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ABSTRACT 

This document begins with a brief overview of what assessment 
is, including a definition, purpose, brief history, and important process 
considerations in creating a successful and effective assessment effort on 
each campus. The three key questions in creating an effective assessment for 
each school are: (1) what do faculty members expect students to learn; (2) 
can it be demonstrated that students have learned; and (3) how can the 
results of the assessment be used to help student learning. There are two 
modes of assessment; direct assessment and indirect assessment. The document 
explores various methods and tools in carrying out assessment of student 
learning outcomes. Four key areas of student learning general education, 
program/ma j or, basic skills, and students and academic support services. 
Issues of access and equity are discussed in the last section of the report 
and are essential components of an effective assessment program. It is 
stressed that assessment is a never-ending process that must provide ongoing 
feedback to help improve student learning and insure student success . The 
document concludes with a list of definitions, a bibliography, including 
websites, and a variety of examples of assessment related materials for two- 
year colleges. Contains 87 references and 7 appendices. (MZ) 
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Prologue 

This Handbook on Outcomes Assessment is written with direct support from a grant 
from the Fund for the Improvement of Instruction (FII) given to College of the Desert by the 
California Community College Chancellor’s Office. , 

The Handbook begins with a brief overview of what assessment is, including a 
definition, purpose, brief history and important process considerations in creating a 
successful and effective assessment effort on each campus. The Handbook then explores 
various methods and tools in carrying out assessment of student learning outcomes (SLO’s). 

The next four sections of the Handbook address four key areas of student learning: 
general education, program/major, basic skills and student and academic support services. 

A fifth section addresses issues of access and equity. 

The Handbook concludes with a list of definitions, a bibliography, including 
websites, and a variety of examples of assessment-related materials for two year colleges. 

It is the intention of the author to keep this Handbook current by adding additional 
new materials and examples as they become available. Suggestions and submittals should 
be sent to the author at: emorante@.coIIegeofthedesert.edu 

This Handbook arose out of discussions surrounding the California Assessment 
Institute ( www. ca-assessment-inst. orgf . The author wishes to thank the members of the CAI 
Task Force for their support and suggestions, especially: Darlene Pacheco (AACJC), Diane 
Ramirez (College of the Desert), Fred Trapp (Long Beach CC), Bill Scroggins (Modesto 
JC), Julie Slark (Rancho Santiago District), Linda Umbdenstock (Long Beach), and Judy 
Walters (Peralta Community College District). 

Thank you also to Maria Sheehan, President of College of the Desert, for her support 
of this effort at our college. Special thanks to Chris Nelson, my colleague and friend at 
COD, and to Peter Ewell of NCHEMS and James Anderson of North Carolina State 
University who took the time and effort to provide very useful feedback and suggestions to 
make this Handbook better. And thanks to Amy DiBello, COD English instructor, for her 
editing, and to Sharon Bentzen at COD for her significant clerical assistance in this and 
many other projects. 

And finally, I want to thank my wife, Linda Morante, for her feedback, ideas, and 
support on this effort, one of numerous times she has helped me in the almost 40 years we 
have been together. 



E.A.M. 
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EXECUTIVE SUMMARY 



The key questions in creating an effective assessment effort on each campus are: 

1 . What do faculty members expect students to earn? 

2. Can it be demonstrated that students have learned? 

3. How can the results of assessment be used to improve student learning? 

Assessment is: 

• the systematic collection of data and information across courses, programs and the 
institution, an integral part of teaching and learning used to help both, and an 
essential component of a college’s mission 

Its Function is: 

• to focus on student learning outcomes (SLO’s), but also includes process, especially 
in seeking ongoing improvement 

• to demonstrate and improve student learning and student success 

• not to serve as personnel evaluation; 

• to facilitate accreditation, accountability and institutional effectiveness 



Modes of Assessment : A comprehensive and effective assessment program should include 
both direct and indirect measures. 

❖ Direct assessment: the measurement of actual student learning, competency or 
performance. Examples include essays, tests, speeches, recitals, capstone 
experiences and portfolios. 

❖ Indirect assessment: the measurement of variables that assume student learning 
such as retention/persistence, transfer and graduation rates, and surveys. 



Four Principal Areas of Learning to Assess : 

1 . General Education: may include such skills as writing, critical thinking, problem 
solving and quantitative analysis as well as such content areas as: arts and 
humanities, mathematics, science and social science 

2. Program/Major: includes the competencies expected of students in achieving a 
certificate/degree beyond basic skills and in addition to general education 
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3. Basic Skills; reading, writing, ESL and mathematics 

4. Student Services/Academic Support Services: includes both the affective 
outcomes defined by a college and the expected outcomes of student programs 
including EOPS, CARE, DSPS, counseling, tutoring, learning center, etc. 

Effective Assessment Programs; 

An effective assessment program should include a component addressing both 
access (how well a college is serving its defined community) and equity (how well a college 
is achieving successful outcomes for different groups of students such as race/ethnicity, 
gender and age). 

Assessment should be implemented at the course, program and institution level. 

Each institution needs to decide for itself where and how to begin. Some institutions will 
benefit from beginning at the course level, while most will probably be better served by 
beginning at the program or institution level. There are advantages and difficulties at each 
level. 



The assessment process must include significant faculty involvement and leadership 
and be strongly supported by administration at all levels. Assessment should be 
institutionalized, therefore ongoing and cyclical in particular aspects. The use of a 
coordinating committee is important, as is the use and integration with such ongoing college 
efforts as; program review, matriculation, institutional research. Partnership For Excellence 
(California), and accreditation (both institutional and program). Creating an assessment 
effort devoid of other ongoing programs and institutional efforts is counterproductive and 
expensive in both time and money. 

To be effective at the institution level, assessment must provide ongoing feedback to 
help improve student learning. Embedding assessment efforts into courses assists in student 
motivation and performance. Utilizing existing data bases, tests (both standardized and 
home grown), sampling techniques and local expertise (e.g. the institutional researcher) are 
important aspects of a successful assessment effort. Taking advantage of what has been 
attempted, learned and accomplished by others is very helpful. 

Like instruction itself, assessment is never ending hard work that is an essential 
component of student learning and student success; it requires extra effort campus-wide. 
Accepting this fact is helpful and motivating. And, once assessment activities are in motion, 
it is crucial to use the results to improve student learning.. 
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A Handbook on Outcomes 
Assessment for Two-Year Colleges 

Edward A. Morante 
College of the Desert 



Introduction 

What do faculty members expect students to know and to learn? How successful are 
our students? How is “success” defined? How can we demonstrate what our students have 
learned? How well have they learned? These and similar questions are at the heart of the 
teacher’s and educator’s mission. Answers to these questions form the essence of what we 
shall call “assessment”. 

This Handbook is a hands-on, down-to-earth, nuts and bolts tool for assessment. 
While this Handbook will include a brief history of the “modem assessment movement,” it 
is not intended as a philosophical treatise, nor as a thorough compendium of assessment 
activities. The Handbook is intended to provide guidelines for assessment, including 
definitions and examples, as well as resources and links to additional information on 
assessment. 

The purpose of this Handbook on Outcomes Assessment is to assist two year college 
faculty and staff to develop, organize, implement and evaluate a successful assessment 
effort, and to put assessment to continuous use to improve student learning and student 
success. This Handbook emphasizes beginning a campus assessment effort at the program 
and institutional level although the definitions, areas to be assessed, processes, and tools are 
included for all three levels. 



What is Assessment? 

The American Association of Higher Education (AAHE) organized a cross-section 
of assessment “experts” under the leadership of Tom Angelo, former director of the AAHE 
Assessment Forum. This task force created the following definition of assessment which 
was widely discussed, debated, critiqued and amended for about a year before consensus 
was reached: 

Assessment is an ongoing process aimed at understanding and improving student 
learning. It involves making our expectations explicit and public; setting 
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appropriate criteria and high standards for learning quality; systematically 
gathering, analyzing and interpreting evidence to determine how well 
performance matches those expectations and standards; and using the resulting 
information to document, explain and improve performance. When it is 
embedded effectively within larger institutional systems, assessment can help us 
to ^ocus our collective attention, examine our assumptions, and create a shared 
academic culture dedicated to assuring and improving the quality of higher 
education. (Angelo, T., 1995, p.7) 

Angelo emphasizes several aspects of this definition: 

1 . The focus of assessment should be on student learning. 

2. Setting expectations, criteria and standards is essential. 

3. Evidence (data) must be systematically collected and used to improve (student) 
performance. 

4. Assessment should be ongoing and institutionalized and become part of the accepted 
culture of the college. 

5. Assessment helps to ensure and improve quality. 

Thus, assessment is important because it helps to demonstrate and improve student 

learning and student success . See Appendix G for the AAHE (American Association for 

Higher Education) Assessment Forum Nine Principles of Good Practice for Assessing 

Student Learning. 



Other Definitions of Assessment 

Assessment also has been used to describe a process of collecting information 
(usually including basic skills placement testing) to determine the reading, writing and math 
skills of entering freshmen. In this basic skills use of the term, assessment is part of a 
comprehensive developmental education program (part of Matriculation in California 
community colleges) and is considered an input measure. In contrast, the definition of 
assessment used in this Handbook has a considerably broader context than basic skills and 
has much more of a focus on outcomes of student learning. 

In addition, assessment has been used as part of the term “classroom assessment” 
originally conceived by Angelo and Cross (1993). Classroom assessment is a series of 
teaching methodologies available to improve teaching and learning within a particular 
classroom by an instructor. For example, a common classroom assessment technique is the 
“one minute assessment” where the teacher at the end of a class asks the students to describe 
such things as: “what was learned most in this class” or “what aspect of this class was still 
most confusing to you”. This information can assist the instructor in better understanding 
both the effectiveness of the class and what area(s) he/she might need to reinforce at the next 
class. In using classroom assessment, no attempt is made to evaluate student learning in a 
broader context - or to grade students. See Bibliography for several references on 
classroom assessment. 
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Institutional Effectiveness 

Sometimes the term institutional effectiveness is used interchangeably and/or confused with 
assessment. Generally, institutional effectiveness is a term used by various components of 
the institution or the institution itself to review how effectively goals are achieved. Sullivan 
and Wilds (2001) provide a useful definition: 

Effectiveness is the evaluation of performance and related actions involving concepts based 
on the following: 

• Accomplishment of the institutional mission 

• A reflection of the institutional vision, philosophy, goals and objectives 

• An interpretation of the environment 

• Data-based historical review of institutional accomplishments, weaknesses and 
aspirations 

• Preparation, collection, and interpretation of data by institutional staff and faculty 

• Involvement of faculty in developing and evaluating the curriculum and in evaluating 
student performance as it relates to the curriculum 

• Involvement of senior line officers in interpretation and use of pertinent data and 
information 

• Leadership of the president in defining and communicating institutional priorities 

• Involvement of the Board in developing and approving policy 

• Success of the president and the board in obtaining resources 

In contrast, as used in The Handbook, assessment will focus almost exclusively, on 
student learning outcomes (SLO’S). expected student performance by the faculty at the 
completion of the course, the program, or the degree/certificate/institution. 



History of Assessment 

Without a doubt, assessment has been a part of higher education from the start. Why 
then the fiiss? Arguably, the current focus on assessment, especially outcomes assessment, 
began with the federal report, A Nation at Risk in 1983. This U.S. Department of Education 
report decried the mediocrity of the American educational system stating that if another 
country had done to us what we have done to our educational system, we would have 
declared war. While this report focused on K — 12 education, it was followed by similar 
reports on higher education raising questions about student learning and the effectiveness of 
our colleges and universities. For example, in 1984, the National Institute of Education 
issued Involvement in Learning which called for increased emphasis on undergraduate 
teaching and learning, concluding: “Institutions should be accountable not only for stating 
their expectations and standards but for assessing the degree to which those ends have been 
met. . . They should make a conscientious effort to acquire and use better information about 
student learning, the effects of courses, and the impact of programs.” (p. 21) 

In 1986, the National Governors Association issued “Time for Results which directly 
connected the interests of state governments in higher education by emphasizing that 
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successful economic development, international competition, school reform, and teacher 
preparation all depend on excellence in undergraduate education. Among the eight 
challenges to the higher education community was the following: “Improve assessment of 
students and institutional performance.” 

A number of states (e.g., Tennessee, Virginia, Missouri) heeded the call and created 
statewide assessment efforts for all public colleges and universities in their states. Across the 
country, many individual institutions (e.g., Alvemo, Truman, James Madison, Kings, 
Sinclair) made assessment an integral part of their teaching and learning. The federal 
govermnent has also significantly increased its focus on assessment, enhancing IPEDS (the 
Institutional Postsecondary Educational Data System, a national databank on student and 
college characteristics), as well as a concerted effort to add outcomes assessment to a variety 
of programs including grants and accreditation. 

In many areas of the country, the principal motivation for focusing on assessment 
activities has come from accreditation agencies, both regional and subject areas. These 
agencies have been spurred by the federal govermnent which in 1989 required assessment 
activities including outcomes to be an integral part of regional accreditation standards. 

Failure to adequately address this issue would jeopardize the agencies’ charter to accredit 
institutions in their region. In turn, the absence of such accreditation would prevent any 
federal funds, including grants and student financial aid, from flowing to those institutions. 
SACS (the Southern Association of Colleges and Schools) was the first regional accrediting 
agency to establish this emphasis on assessment more than 15 years ago. 



California Community Colleges 

Today, there are at least three primary external forces impacting assessment in 
California community colleges: the State, the regional accreditation agency, the Accrediting 
Commission for Community and Junior Colleges (ACCJC) and the California Assessment 
Institutes. Several years: ago, the Chancellor’s Office for the Community Colleges reached 
an agreement with the governor and the legislature to create Partnership For Excellence 
(PFE). This agreement provided additional state funding to all community colleges in 
California based on FTES in conjunction with achievement of system- wide output 
improvements over time in five areas: basic skills, course completion rates, graduation rates, 
transfer rates, and economic development (workforce training). The emphasis on system- 
wide results in the current law calls for statewide improvements, while seemingly, holding 
harmless any individual institution. 

ACCJC requires, in its current standards, that every institution create “a climate of 
evidence” as part of its accreditation self-study. Recently, ACCJC has revised accreditation 
standards increasing the need to carry out assessment and provide data and other evidence of 
student learning and institutional effectiveness. These new standards, which will go into 
effect, beginning with institutional self-studies in 2004, place greater emphasis on 
demonstrating outcomes. These new standards also bring ACCJC closer to those standards 
required by other regional accreditation agencies. (See the Southern Association of Colleges 
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and Schools (SACS) and The Higher Learning Commission (North Central Association of 
Schools). 

The California Assessment Institutes (CAI) provide a third major emphasis for 
assessment activities at California community colleges. In 1997, a group of interested 
faculty and staff from several colleges and constituencies began hosting a series of statewide 
Institutes on assessment. Funded by the Chancellor’s Office and attendee registration fees, 
and supported by ACCJC, the Research and Plaiming (RP) group and several California 
community colleges, these institutes have focused on helping faculty and staff understand 
what assessment is and can be. This Handbook on Outcomes Assessment is one direct 
outcome of these CATs intended to provide, in writing, a tool for understanding assessment 
and developing and implementing a successful assessment program on campus. 

Student Learning Outcomes (SLO’S), Domains and Rubrics 

As with any endeavor, assessment has its own terminology. Many definitions are 
included throughout this Handbook with a compilation listed on pages 38-43. Three terms, 
in particular, are essential for understanding what assessment is and how to implement a 
successful effort. These are: 

Student Learning Outcomes (SLO’s): the competencies and skills expected of students as 
they complete a course, program or institution. 

Domain: a set of skills or subskills in a particular educational area; for example, the specific 
skills that make up algebra or critical thinking. 

Rubrics: a set of scoring guidelines for evaluating student’s work. Rubrics make explicit 
the standards by which a student’s work is to be Judged and the criteria on which that 
Judgment is based. (See Appendix D) 



Methods of Assessment 

Assessment methods can be divided into two main categories: direct and indirect. 

A. Direct Assessment is the measurement of actual student learning, competency and 
behavior. Lopez (1999) listed several examples of direct measures of student 
learning: 

■ capstone experience 

■ portfolio assessment 

■ standardized tests performance on national licensure, certification or professional 
exams 
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■ locally developed tests 

■ essay questions blind scored by faculty across the department, division or college 

■ qualitative internal and external juried review of comprehensive senior projects 

■ externally reviewed exhibitions and performances in the arts 

■ external evaluation of performance during internships based on stated program 
objectives 

Lopez states that “direct measures of student learning yield useful information about 
the value added to a student's learning . . . especially when the results from multiple 
measures are triangulated and are compared with (1) baseline data and/or with (2) data from 
other measures taken over time.” (Lopez, 1999) (See the Definitions section below for more 
information on “value-added”.) 

Indirect Assessment examines student performance or behavior using criteria 
which, if accomplished, assume learning has taken place. Lopez (1999) states, “many 
sources of data, if used alone, are inadequate measures of student learning. However, some of 
these sources, when used to supplement direct measures, provide information that may enrich 
or illuminate aspects of what the direct measures tell us about students' academic 
achievement.” She lists the following as examples of indirect measures of student learning: 

• alumni, employer, and student surveys; 

• exit interviews of graduates and focus groups; 

• graduate follow-up studies; 

• retention and transfer studies; 

• length of time to degree; 

• entering test scores; 

• graduation rates and transfer rates; 

• job placement data. 



Both direct and indirect measures of assessment should be used for an effective 
assessment program. 



Purpose of Assessment 

The principal purpose of assessment is to demonstrate and improve student learning and 
student success. To this purpose, assessment can: 

1. Improve how well and what we teach - and what we expect students to learn; 

2. Provide information for improving learning; 
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3. Focus on process as well as on inputs, outputs and outcomes; 

4. Actively involve faculty, staff and students; 

5. Use multiple and varied measures; 

6. Be carried out at various key points in the educational process; 

7. Provide feedback to those most affected; and 

8. Be an intrinsically educational activity. 



Assessment Process 

While the details may differ in each area of student learning, the basic process of 
assessment generally consists of five key components: 

1. Intended Student Learning Outcomes (SLO's): What should students know and be 
able to do when they complete the course, program and institution? These need to be 
specifically defined in terms of both content and skills. 

2. Criteria for Success: What are the standards by which these SLO’s will be judged? 

3. Assessment Tools: What devices and methods are used to measure attainment of the 
SLO’s? As in the other areas of learning, both direct and indirect measures should 
be used. For specific examples, see the section below on Assessment Tools. 

4. Assessment Results: What are the assessment findings? 

5. Use of Results: What changes will be made as a result of faculty reflection on the 
assessment findings? What areas of concern need attention to promote student 
learning and departmental and institutional effectiveness? 



The following chart provides one model for creating an effective campus assessment 
process: 



ERIC 
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Achieving Success in Developing Assessment 

There is no one method for creating, implementing and institutionalizing an effective 
assessment program. Based on what other colleges have experienced, the following points 
seem to increase the probability of success: 

1. Institutional Commitment 

The more the institution as a whole is committed, the more likely any effort, 
including assessment is likely to be successful - and to continue beyond changes 
in leadership or key personnel. 

2. Leadership from the Top 

There is much said about how important it is to have ideas come from the ground 
up. And while grassroots involvement is usually essential (see the next point), 
rarely do major initiatives succeed without leadership from the President, Vice- 
Presidents, Academic Senate, and other key leaders. There are too many very 
real obstacles to a successful assessment effort without adding bureaucratic 
obstruction. There will be many opportunities for leadership from the top to 
unlock doors and to find sufficient funding. Leadership from the top provides 
role models, reinforcement, commitment and often additional incentives 
important in mtiking changes and institutionalizing those changes. Public 
commitment by the Board of Trustees and periodic reports to them also improve 
the likelihood of success. 

3. Broad-based Input 

This complements leadership from the top. It is very unlikely that a top down 
approach alone can be successful. Ideas can and frequently do come from a 
variety of sources. Creating an atmosphere of trust often leads to creative 
thinking and problem solving. The goal is to achieve consensus, remembering 
that consensus does not mean unanimity. Dialogue should be encouraged rather 
than advocacy of one’s point of view. There must be a balance involving 
compromising, risk taking, solution seeking and disallowing stonewalling. 

4. Communication 

Open communication involving all key constituencies is essential. Agendas 
should be drafted at the end of previous meetings and published well ahead of the 
next meeting to allow adequate input and preparation. Minutes and attendance 
taken and published well ahead of meetings also helps to keep appropriate 
records, motivate members and prepare for future discussions. Schedules and 
calendars discourage conflicts and decrease missed meetings. 

5. Start Small and Be Practical 

Build on what is already available, including an inventory of what is currently 
being done on campus. Be incremental and don’t try to do everything at once. 

6. Patience. Perseverance. Time 
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Worthwhile programs frequently take time to get organized and perseverance and 
patience to be successful and become a part of the culture of a campus. The 
history of assessment demonstrates that these attributes are essential for success. 

7. Data: Multiple Variables 

Very rarely does one variable or a one-time effort succeed in providing reliable 
or valid information. Fault can and will be found with every variable, method or 
instrument used in assessment. The use of multiple variables, such as viewing 
things from different perspectives, is crucial for analysis and reaching 
conclusions. Similarly, replications help to confirm (or revise) conclusions. 

8. Results 

While assessment should be an ongoing process, there is also a time to examine 
results. This process should be seen as taking a snapshot at particular points in 
time. At first, semester results might predominate, then annual, and eventually 
periodical. One model might include reporting on different areas in different 
years in a cyclical approach. Trend analyses of changes over time can also be 
extremely helpful in demonstrating whether change has taken place and where. 
The importance of discussion and interpretation of the results, especially among 
faculty, must be emphasized. 

9. Reports: Feedback 

Periodic broad reporting of results is essential for many reasons including: a 
sense of accomplishment by participants; a reinforcement of trust and openness 
to the campus community; availability of useful information for plarming and 
decision-making; revisions, changes and improvements where needed; and 
fulfilling needs to demonstrate learning and student success. 

10. Development & Revisions 

Assessment is and must be an ongoing process of institutional change and 
improvement to achieve appropriate outcomes of student learning and student 
success. Assessment is NOT a giant research project. In this regard, from a 
classical research perspective, assessment is only formative (make improvements 
as you identify a need for change) and not summative (issuing a final report). 
Improving outcomes is paramount, ongoing and subject to change throughout. 

11. Institutionalization 

An institution has a successful assessment program when the process and impact 
of assessment is well understood by students, faculty and staff alike. It is 
important to seek ongoing improvement, a culture of using the results, and a 
climate where assessment is viewed as essential, even fim. Assessment is part of 
instruction; learning includes assessment. 

Course, Program and Institutional Assessment 

A comprehensive assessment of SLO’s should be developed at the course, program 
and institutional level. Each level provides some perspective of successful achievement of 
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outcomes in the principal areas of learning described below. The institution must decide 
how best to accomplish its outcome goals. 

Some institutions have found that assessment efforts are easier to implement at the 
course level, especially where there is a tradition of defining course objectives in a 
measurable way that includes SLO’s. Many faculty are comfortable initiating the 
assessment process by measuring outcomes in the courses they teach. 

On the other hand, Nichols (2002) pointed out that most institutions that have begun 
with assessment at the course level tended to get bogged down in minutiae. Further, when 
faculty begin at the course level, there often appears to be greater difficulty in making the 
transition to the broader concepts/perspective needed for general education, program and 
institutional assessment. 

Each institution must decide where to start assessment according to its own unique 
qualities, history and environment. Beginning at the program or institutional level 
encourages review of broader issues beyond the individual classroom. The broad view 
permits development of the larger picture and definition of expected SLO’s across 
disciplines. From the larger perspective, the curriculum can be reviewed and revised as 
needed. The specific courses and their outcomes need to be integrated into the revised 
curriculum. 



Assessment Tools 

An important step in assessing SLO’s is to select the methods and/or tools to be 
used. Tests, standardized or locally developed, are sometimes the easiest to choose for this 
aspect, but they are not the only methods of assessment. 

1. Using Tests in Assessment 

Tests are the most common means used in education to assess student learning. They 
come in many forms: standardized and home grown, objective (e.g., multiple choice, 
true/false, fill-ins) and non-objective (e.g., essay, performance). For the record, 
standardized tests are NOT all multiple choice tests but can be performance-based such as 
an essay, a speech, or even a series of tasks requiring the student to respond to a variety of 
stimuli with a variety of responses. 

Almost all instructors use tests in helping to determine student learning and 
achievement. To a greater or lesser degree, tests play important roles in awarding grades, 
traditionally the most common means of characterizing student learning. Most of these tests 
are home grown, developed by individual faculty members based on their judgments of the 
course syllabus, what has been taught and what is expected to be learned. Most faculty 
consider development of such course tests to be their unique prerogative, akin to academic 
freedom. Most faculty also consider the process of test development relatively easy to 
implement. 
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In contrast to popular lore, developing reliable and valid tests that meet high 
standards, including avoidance of bias, is a complex process. It includes careful 
conceptual/domain development; the expertise of writing appropriate, unambiguous 
questions/items; the testing of these items for reliability and validity across different 
populations; the necessary revisions; and the setting of standards (norm-referenced or 
criterion-referenced) that makes a test “standardized”. Very, very few faculty have the time 
or have the psychometric expertise to write, review, revise and determine the effectiveness 
of their tests. An example of such difficulty can generally be observed in producing a home 
grown survey. In many ways, development of a survey and a test has some common 
characteristics - and difficulties. Many educators have developed a survey. It begins with 
conceptual thinking of at least both the domain and the population that is to be surveyed. 
Questions are written, reviewed, and revised before administering the survey. Too often in 
this process, we are disappointed about the “misunderstandings” that some people have with 
some of our best efforts. How could they have answered that question in that way! How 
could they have misinterpreted what had been so carefully written? A well-constructed, 
valid survey is not easy to produce; a valid test is generally even more difficult. 

It is beyond the scope of this Handbook to adequately discuss psychometrics or test 
development. It is fair to state that tests are widely used and widely misunderstood. There 
are advantages and disadvantages in both home grown tests and standardized tests and these 
factors should be considered carefully in developing an effective assessment effort. Nichols 
( 1998 ) describes the advantages of using home grown tests including: 

a. They are more readily accepted by faculty; 

b. There is greater likelihood of faculty using the results, at least initially; 

c. They are often easier to design a fit for local curriculum/program. 

The disadvantages include: 

a. Time and effort of faculty to construct a reliable, valid and unbiased test; 

b. Lack of external credibility; 

c. Lack of opportunity to make comparisons of student performance outside the 
institution. 

The advantages and disadvantages of using standardized tests include: 

Advantages: 

a. They are more likely to be both reliable and valid; 

b. They provide comprehensive coverage of the subject area/domain to be assessed; 

c. The ready availability of such instruments saves much time and effort; 
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d. They allow for comparability of student performance outside the institution; 

e. There is greater acceptance of results by those outside the institution. 

Disadvantages: 

a. There is frequent hostility of faculty to standardized tests; 

b. The reluctance of faculty to use the results; 

c. The possible disparity between test and curriculum. 

An additional factor to consider is cost, although this may be initially deceiving. 
Obviously, the use of a standardized test requires a purchase from a publisher necessitating a 
budget that must be planned for and is relatively easy to eliminate in times of financial 
difficulties. In using a home grown instrument, the cost of faculty time in development can 
be extensive and can lead to a greater emphasis and effort on test development than on 
analyzing and using the results to improve learning. The focus (some would say excessive 
emphasis) on evaluating placement tests at the local level, as is currently required of 
California community colleges, provides a very real example of consuming time and effort 
on test development at the cost of improving what is learned by students in basic 
skills/matriculation efforts. 

Licensure. Certification and Professional Examinations are standardized tests that are 
sometimes available, especially in the health sciences, and can be used as an assessment 
device for student learning. These kinds of tests are most useful for the program major and 
not for general education. Appendix C includes a brief description and references for the 
leading standardized tests that are being used by colleges across the country to assess 
general education. 

In addition, a number of other devices and methods besides standardized tests are 
being used by a variety of institutions to assess student learning in basic skills, general 
education and program/major. These methods include: 

2. Departmental Exams 

These were common years ago at many colleges and universities and appear to be 
having a revival. They are devised as common exams by a department faculty based on a 
consensus of concepts that are in the syllabus of one or more courses and are expected to be 
taught, and thus learned, regardless of which section a student enrolls in. They seem 
especially useful for those courses that are either requirements for many, if not all, students 
or have heavy enrollments of students. The advantages of such departmental exams include 
a consensus of both concepts that must be covered by the faculty and standards that must be 
met by the students. A disadvantage arises when not all faculty cover all domain aspects in 
a particular section or don’t cover them well causing students’ performance to suffer. This 
latter issue has been addressed in some locations by having the instructor grade the 
departmental exam for his/her students as part of the grading process for those students in 
that particular section, while a departmental group may score the exams a second time for 
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assessment purposes. This separates, to some extent, the grading process from the 
assessment process as advocated by some. It also addresses the concern about motivating 
students to take the assessment seriously and perform their best. 

Using a sampling procedure can simplify this process. For instance, a random 
sample of students could be selected, or only some parts of the departmental exam could be 
selected for rescoring. Either option could provide adequate data for assessing student 
competencies/leaming in a program or institution, without impacting the grades of 
individual students. (See also course-embedded assessment below.) 

Warning : In setting standards, take care not to set them at an unrealistically high 
level based more on expectations (hope?) than on reality. 

3. Course-embedded Assessment 

As the name implies, this is a process of carrying out assessment within the normal 
class or course. There are at least three kinds of assessment that can be implemented within 
the current course structure of teaching, learning, grading and assessment/evaluation: 
classroom assessment, course assessment and course-embedded assessment. All three of 
these assessment activities must include very strong faculty initiation and involvement. Not 
all of these are formal structures that must be documented or provided to external sources. 
Classroom assessment, (for example, Angelo and Cross” Classroom Assessment 
Techniques or CAT’s) as was described earlier in this Handbook, is a series of techniques 
and methodologies which any instructor could employ to assess how well a course is going, 
help determine how well students are understanding the concepts being covered, identify 
areas that might need reinforcement, etc. In this sense. Classroom Assessment can be a very 
helpful device for providing ongoing information about how a course is progressing, but it is 
not sufficient for assessing how well students have learned or can demonstrate proficiency. 

In addition, of course, more formal (and traditional) procedures such as quizzes and 
tests provide feedback on how well students are learning, the results of which can provide 
feedback to the instructor on what, if any, mid-course corrections might be needed, concepts 
reinforced, methods altered to improve student learning. Information from quizzes, tests and 
classroom assessment activities should be used to provide feedback to the students in the 
course as well, including having a chance to discuss the test results wherever appropriate. 
This feedback can also help students considerably in their learning by encouraging them to 
focus on areas needing improvement and reinforcing the important concepts of the course. 
These are some of the activities that should happen in every course, virtually regardless of 
an assessment effort beyond individual courses. In this regard, testing and assessment can 
directly improve student learning, not merely evaluate the success of achieving outcomes. 

Course assessment/evaluation is also a well-respected and traditional method in all 
levels of education and includes setting course goals and objectives, adding expected SLO’S 
(all included in the course syllabus), and then evaluating whether these outcomes were met, 
how well, and by how many. Adding the outcomes to the course syllabus includes defining 
the expectations of what the students are expected to learn or perform by the end of the 
course, and then assessing how well the students have achieved the competencies or skills 
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expected. This kind of course assessment strengthens the grading process by increasing the 
delineation of outcomes and standards expected and used to evaluate student performance, 
regardless of instructor. Classroom-based Assessment (Walvoord, et al, 1998, etc.) provides 
a specific model that ties into both traditional course evaluation techniques and adds a strong 
SLO assessment component/model. 

Course-embedded assessment is the use of a course and/or the classroom to • 
implement program and institutional assessment activities, such as assessment in general 
education or the major that extends beyond the individual course. This is a method of 
sampling that allows broad assessment activities to be carried out within the course structure 
instead of, or in addition to, creating a separate assessment process, such as an Assessment 
Day. For example, a number of questions that can be objective, short answer, or essay can 
be added (embedded) to the regular course final exam. These additional questions are then 
scored and used for program and institutional assessment. (Because each student may 
receive different questions, this method does not permit individual student assessment but 
these additional questions do not interfere with the normal grading system for the course.) 

Course-embedded assessment offers a simple way to implement assessment activities 
for general education, program/major or basic skills with greater likelihood of both faculty 
and student involvement. Because the assessment activities are “embedded” within the 
course content, syllabus and assessment/grading practices, not separated from it, this method 
also helps students to be motivated to perform to the best of their abilities. Course- 
embedded assessment should be a meaningfiil part of the course, realistically applied and 
used in the course. The embedding of course objectives should also take place in homework, 
assignments and similar projects to facilitate learning and provide practice for the course- 
ending assessment tasks. 

The results can be documented both by the instructor for the students in that course 
as well as for the larger picture across the institution, program or multiple courses. Course- 
embedded assessment involves external collaborators (e.g., other faculty members, 
employers, etc.) who are involved, with the course instructor, in assessing a broader context 
of student learning beyond the individual course level. 

A compilation of several other course embedded assessment methods was developed 
at Kirkland College including: 

Ill-defined or Ill-structured problems. An ill-defined problem is one that is not highly 
structured and cannot be resolved with a high degree of certainty. Experts may disagree 
about the best solution. Examples: determining what really happened at Waco or solving the 
nuclear waste storage problem or predicting the effect of global warming or deciding if there 
is such a thing as global warming. Dealing with ill-defined problems requires the integration 
of many skills, abilities, and areas of knowledge. 

Critical Incidents. Students can be asked to describe an incident, either real or imagined, 
that illustrates or illuminates key concepts or principles. An explanation of the concepts or 
principles illustrated should accompany the description of the incident. 
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Writing Samples. Writing assignments can be used as a measure of students' mastery of 
course content and attainment of program or major goals. Such assignments may also be 
used as a measure of the general education communication skills goal as well as an 
assessment of critical thinking skills. Examples of writing samples include essays, research 
or term papers, answers to essay questions on tests, book reports, summaries, lab reports, 
and the like. Writing samples can be graded holistically using a rubric. (See Appendix D 
for examples of scoring rubrics.) 

Oral Presentations/ Oral Exams. Depending on the nature and content of the course, oral 
presentations can be tailored not only to assess students' mastery of course content but also 
their attainment of general education goals such as critical thinking, general knowledge and 
historical consciousness, understanding the impact of science and technology, and 
educational and social values. Oral presentations based on course content can be used as a 
direct measure of students’ communication skills. 

4. Capstone Experience/Course 

A Capstone Experience is a course, intemship/fieldwork, or project that provides an 
opportunity for students to demonstrate their ability to synthesize what they have learned 
from a variety of courses. It is the culmination of a program or a degree to demonstrate 
readiness to work in the field and/or to continue on to higher levels of education. Examples 
are more common with majors or vocational programs: creating a project model in 
architecture; writing a computer program or debugging a computer problem; performing a 
recital or putting on an exhibit; identifying possible causes of a problem and fixing it (e.g., 
an auto, an air conditioner, a motor) and so on. In general education, a capstone experience 
is usually a course or experience where students are required to draw from the knowledge 
they have gained from a variety of experiences and courses, organize and synthesize the 
knowledge and skills developed, take and defend positions, both orally and in writing, and 
perhaps prepare a report that includes data to support recommendations. The development 
and assessment of a capstone experience is usually a joint effort of the faculty in a program 
or, in the case of general education, a cross-section of faculty from a variety of disciplines. 
The capstone experience, including the structure and content, needs to be clearly linked to 
the purposes and objectives of the program or institution, including general education, and 
what the students need to have learned in order to graduate from the program or institution. 

5. Performance Assessment 

Performance assessment is an assessment method that requires students to perform a 
task rather than take a test. Performance assessment is designed to judge students' abilities 
to use specific knowledge and skills and actively demonstrate what they know rather than 
recognize or recall answers to questions. Performance assessment is sometimes called 
authentic assessment because it involves tasks in an authentic or real-life context or a 
context that simulates a real-life context. (See Wiggins, 1998) Examples of performance 
assessment in General Education include an analysis of videotape recordings of speeches, 
analysis of written prose, essays, term or research papers, responses to case studies, or 
formal presentations. 
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In some courses, opportunities can be found to ask students to engage in a simulation 
of a real-life problem that they must solve using the knowledge and skills they have gained 
in the course. A single project can be structured to assess both mastery of course content 
and attainment of program or major goals as well as certain general education goals such as 
communication skills, life-long learning skills, critical thinking skills, and social and 
education values. For example, students might be asked to assume the role of a city council 
member who must make a decision concerning a controversial issue. Students might then be 
asked to research both sides of the issue and to deliver a persuasive speech or to write an 
action plan. 

Another example of performance or authentic assessment is the use of a case study. 
Students are presented with a realistic example of an application in the field and must 
respond with an analysis that demonstrates mastery of course content and ability to apply the 
information and skills learned. A case study is an examination of a specific phenomenon 
such as a program, an event, a person, a process, an institution, or a social group. The end 
product of a case study is a robust description of the phenomenon being studied that 
demonstrates the student's understanding of the phenomenon through the application of the 
knowledge and skills they have learned. 

6. Portfolios 

Portfolios are collections of samples of each student’s work designed to demonstrate 
and assess a student’s learned performance in one or more areas. These have been 
traditionally popular in both academic and work realms in the arts, design, architecture and 
similar fields where it is necessary to display talents and proficiencies. Increasingly, other 
areas, especially English/writing, have begun using portfolios to demonstrate both 
performance and growth over time in the area under review. As with other methods of 
assessment, relevant links need to be established between the portfolio analysis and the 
purposes and objectives of the domain, program or institution being assessed. When using 
portfolios, it is also very important, and frequently difficult, to develop assessment protocols 
as to how the portfolios are to be reviewed. These protocols should include what materials 
will be placed in the portfolio, how these materials will be assessed, by whom, with what 
standards and at what time intervals. It is important to determine whether the materials in 
the portfolio are examples of the student’s best work or typical work. Was the work done in 
the classroom or as a result of a course or program or can any work or material developed by 
the student outside of the institution be included? 

Portfolios should be assessed in each area using commonly agreed to, and written, 
procedures or rubrics similar to the kind of consensus needed for departmental exams. 

These rubrics must be clear and need to be shared with students prior to including their 
work. Electronic portfolios (e-portfolios) are a relatively new phenomenon begiiming to be 
used by a number of institutions. 

7. Surveys for Affective Learning (Attitudinal Development) 

Surveys are probably the most common type of measure used by institutions to 
measure the attitudinal development of their students. Although surveys are not sufficient 
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evidence of cognitive learning, they can provide useful information in determining change 
and growth in what students have gained in the affective domain. Examples of surveys 
include: Student Satisfaction/Attitude Surveys, Alumni Surveys, and Employer Surveys. 
Perceptions of student learning can be helpful in making programmatic changes but should 
not be used as the sole or principal basis for assessing student learning. Focus groups 
provide an alternative and/or supplement to surveys and could be employed in a similar 
manner in assessment. 

Logistical Considerations in Assessment of SLO’s 

There are a number of issues and options that colleges have in carrying out an effective 
assessment of SLO’s. 

1. Assess the entire population or a sample: Colleges have a number of choices in 
whether to assess an entire population or some sub-set: 

a. If students are required to demonstrate competency in order to achieve program 
completion, certificate or degree, then, obviously, every student will need to be assessed in 
those areas. For example, if a college requires demonstration of writing competency for 
graduation, all students seeking a degree would need to be assessed in writing. 

b. Alternatively, the college might have a policy that successful completion of a 
required course or courses in, for example, college composition might be the standard set for 
demonstrating competency in writing. In developing an effective assessment system, this 
college might choose a sample of students selected randomly from those who had achieved 
the set standard (e.g., a “C” or better in the selected course(s)). This sample of students 
would then be directly assessed in writing to determine whether the successful completion of 
a particular required course was sufficient to ensure that students had achieved the writing 
competency at the time or just prior to graduation. 

In this sampling procedure, the emphasis is not on demonstrating that each student 
had achieved competency, but rather that the institution’s standards were appropriately set, 
and produced students who could actually demonstrate writing competency regardless of 
which section of the course the student had enrolled in. In assessing either the whole 
population, or in sampling a subset of that population, the college could, and should, use the 
results, as needed, to make appropriate changes in the program to improve student 
competency. In using sampling procedures, special care must also be taken in to assure 
student motivation. (See below.) 

These examples for writing can be extrapolated to other areas of competency required for 
graduation . 

Another alternative in assessment is for a college to choose to assess different 
students for competency in different domains. Either the full population or a sample could 
be used, but, of course, this procedure would be for assessing the institution (or program) 
and not the individual students. For example, let’s assume the institution wants to assess 
how well its students are able to demonstrate their competency in such areas as writing. 
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quantitative reasoning, critical thinking and several areas of content in general education 
(social sciences, humanities, etc.). The college could use a sampling procedure that allows 
different students to take only certain aspects of a total assessment effort. This could be 
done in a variety of ways including some students being assessed in one or more areas while 
others are assessed in different areas. More likely, each student would be assessed in a 
format of random selection of areas assessed. It would be analogous to a large test where 
different students were randomly selected to take each part. Again, the results could be used 
to appropriately identify areas needing improvement that, in turn, could help identify 
methods, etc. that would lead to improvements in student learning. 

Sampling methods allow for an assessment of a broad spectrum of areas at 
considerably less cost than assessing everyone on the full range of possible domains. The 
trade off, of course, is that sampling precludes using the assessment results for assessing 
individual students in those domains sampled. It is also possible to combine different 
sampling techniques such as assessing everyone for competency in one or more areas and 
also assessing different domains randomly across students as described. 

2. Choose the time for assessment activities. In general, the assessment should take 
place at the end or near the end of a student’s completion, whether it is a course, program or 
the institution itself. However, when assessment is carried out on a course basis only, there 
is the likelihood that students will be able to treat the curriculum as isolated entities, falsely 
concluding that when a course is completed, they can, in effect, turn off a portion of their 
brain and “never have to deal with that stuff again!” Such an attitude is antithetical to the 
purpose of providing a general education curriculum. 

If assessment of SLO’s is to demonstrate that students who complete the curriculum 
can effectively relate and communicate with a broad diversity of people and can think, 
synthesize and solve problems in a variety of modes pulling from an assortment of skills and 
content, it may be necessary to set the time for assessing a particular skill well after it is 
actually taught in a primary course. For instance, a student may learn the techniques of 
writing strong research papers in English, but the assessment of that skill might take place a 
semester or longer later, possibly through a portfolio assessment or as part of a broader 
assessment effort that includes a spectrum of skills and content. This method provides 
somewhat greater assurance that what is sometimes called “deeper learning” has taken place. 
The same comments would hold for assessment in general education, program/major or 
basic skills. 

A number of institutions (Appalachian State University in North Carolina, for 
example) have set aside an Assessment Day for students to be assessed. Each student is 
assessed in some area, randomly selected across a variety of domains and assessment tools 
including surveys. This method of assessment is seen as a part of the culture of the 
institution: assessment is an important, integral part of the education provided as is the 
college’s intent to use the results to improve student learning. 

The type of assessment program developed by each college reflects the 
philosophy of that institution. 
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3. Address student motivation. This can be a crucial aspect of an assessment effort and 
MUST be carefully considered and addressed. If students do not take an assessment effort 
seriously, the results can be meaningless or worse, provide misdirection. An example of a 
well-used assessment of student learning occurs when nursing students take their Nursing 
Board examinations. To fail means more than discouragement; failure means inability to be 
licensed. Too many students failing this exam also reflect, to a large degree, on the program 
and institution where they received their education and their training. There is little doubt as 
to the motivation of the students (and the faculty) for successful achievement on this exam. 
The message is clear: the greater the meaning of the results, the greater the motivation to 
achieve success and to want to demonstrate competency. On the other hand, too much 
pressure can lead to anxiety, and a likely decrease in performance. Embedding assessment 
within the normal course structure seems to be a very effective method of ensuring 
appropriate student motivation for many areas and aspects of assessment. Embedded 
assessment methods also seem to be the least likely to disrupt the learning process - indeed, 
it should enhance it. 

4. Communicate positive institutional messages . In examining successful assessment 
efforts across the country, an important component seems to be the messages that 
institutions give to students, directly and more subtlety. When the institution and its 
members, with active participation and leadership by the faculty, including the Academic 
Senate, and strongly assisted and supported by the college’s President, backed by the Board, 
view assessment as an essential component of student learning and instruction, of student 
success, than the college including the students are much more likely to cooperate and to 
buy into the program. This seems to be especially the case when assessment is seen as being 
a normal aspect of what the college does. The more institutionalized, the more assessment is 
an integral part of the culture, mores, climate of the teaching/leaming process, and the more 
accepted it is by students, the more beneficial it is to all well-established assessment effort is 
integral to a learning-centered college because assessment enhances, improves and 
demonstrates learning. 

5. Use and distribute results. Part of acceptance and institutionalization of assessment 
efforts is directly involved with how the results are used. The results should be used to 
improve student learning including assisting in making appropriate curricular and 
pedagogical changes. An assessment program that does not focus on and achieve 
improvements in student learning is worthless. In this regard, the use of assessment 
results should be carefully considered by the faculty. It is also important to communicate 
that improvement came about as a result of planning for and implementing assessment. 

The use and reporting of results must be as carefully planned as the initial 
development of an effective assessment effort. Issues of what should be reported, to whom, 
by whom, how often, etc, should be weighed and evaluated. In general, students who are 
assessed are owed the right of feedback on their performance. This is especially true when 
results are used to assess individual skills and competencies. When sampling procedures are 
used, the minimal report to those who participated would include a summary of the results. 
When sampling is used, care must be taken when providing results on an individual basis to 
ensure that the feedback is accurate and representative of the individual student. 
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The results should also be made available to those who participated in the assessment 
efforts and to others who could benefit from them. In this regard, assessment should 
become an integral component of program and institutional planning, budgeting, and staff 
development activities. Assessment, as well as accreditation, should be seen as an ongoing 
effort for improvement and learning. 

The results of an assessment effort, including the various aspects included in 
this Handbook , should NOT be used to evaluate faculty or other personnel. Anderson 
(2002), however, describes the use of a Teaching Portfolio which can be used in conjunction 
with faculty development. The Teaching Portfolio is a common set of materials and work 
samples that include a selective portrayal of a teacher’s work, not an accumulation. It 
indicates self-reflection and improvement, provides authentic evidence of teaching 
effectiveness and student learning and indicates the pedagogical reasoning or thinking 
behind one’s teaching performance. Anderson divided the Teaching Portfolio into six parts: 

1 . T caching responsibilities 

2. Reflective statement of teaching philosophy/goals 

3. Representative instructional materials 

4. Evidence of student learning 

5. Recent evaluations 

6. Description of activities to improve teaching. 



Principal Areas of Assessing Student Learning 

This Handbook addresses four principal student learning areas for carrying out student 
learning assessment activities: General Education, Program/Majors, Basic Skills and 
Student Services/Student Support Services. A fifth section addresses issues of Access and 
Equity. 



I. GENERAL EDUCATION 



Student learning in General Education is arguably the most difficult to assess. The 
difficulty begins with defining what is meant by “general education”. Across the U.S., 
every college defines this differently, but what makes the definition especially difficult is 
that there is often little agreement on an individual campus as to what constitutes General 
Education. While students are frequently expected, or even required, to take courses within 
designated categories of courses (e.g., communications, written or oral, math and sciences, 
arts and humanities, social sciences), there is less agreement across an institution as to which 
courses count or fall into which categories. 
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Different philosophies and definitions, as well as politics add to this confusion. For 
example, inclusion of courses among General Education categories has a powerful impact 
on enrollments, i.e., money and power. At some colleges, it is difficult to find a course that 
isn’t included on some General Education list. Moreover, even when consensus is locally 
reached, analysis of student transcripts often demonstrates that students regularly follow 
paths different from what the faculty perceives as their definition of General Education. 
(Ratcliff, 1993) Research has shown that, in contrast to the widely held belief that General 
Education should help lead to a well-rounded, generally educated person, students have 
strong tendencies to take courses that strengthen their strengths and avoid their weaknesses. 
(Gaff, 1991) 

Before addressing the “what” of General Education, assessment in this area of 
learning, to be effective, must be carried out at the program level and, more likely, at the 
institution level. Assessment of General Education only at the course level fails to 
adequately address the development of learning across courses. A critical goal of General 
Education must be a broad synthesis of learning that goes beyond individual courses. In 
addition, assessment at the program and institution level permits greater consideration of the 
“whole person,” treats General Education comprehensively, and addresses the requirements 
of accreditation to move beyond the individual section and instructor level. 

Assessment of General Education requires cooperative work across department and 
division boundaries. While this may increase difficulties in reaching consensus on 
definitions and development or selection of assessment strategies and instruments, it also 
offers the potential for creating an integrated educational program based on learning. The 
creation of General Education outcomes, based on what students can do upon completing 
the curriculum, rather than the activities contained within individual courses or academic 
departments, is not easily achieved. However, most faculty who have gone through this 
process, have found it to be enlightening, intriguing, stimulating and even exciting. 

What Should be Assessed in General Education? 

The answer to this question leads to two main paths: content and skills with 
behavior as a third possible area. For clarity, we will deal with each of the two main paths 
separately, but keep in mind that they are NOT mutually exclusive. 

a. Content 

One solution for two year colleges in defining the “what” or content of General 
Education is to examine the requirements or guidelines provided by universities for their 
baccalaureate degrees. For California community colleges, the guidelines provided by the 
University of California (IGETC) or those of the California State University system provide 
a model. Faculty may select either the principal areas such as communications, math, 
science, arts and humanities, social sciences, or the underlying concepts expected to be 
learned by all students in each of these main components. The assessment, then, would be 
conducted in each of these principal component areas, although it need not be done all at 
once, or even with all areas for all students. 
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The faculty at each institution determines the SLO’s to be assessed depending on 
agreed upon definitions. For example, in determining a reasonable assessment of a General 
Education science component, faculty at one institution could decide that, for all students to 
achieve a degree, they need to know certain scientific principles. This determination could 
be reached regardless of specific science courses listed as options for students to complete 
the General Education requirements for science (both physical sciences and biological or life 
sciences) or which courses they actually completed within the approved options given. Once 
this science component is defined, the curriculum in all of the option courses could be 
reviewed to ensure that the identified scientific principles were taught in all the courses 
listed by the college as available to meet the science General Education requirements. 
Assessment would then be made of how well students had learned these principles. This is 
also a good example of how the assessment follows the conceptualization of expectations, 
the definition of SLO’s. rather than the other wav around. 

At another institution, instead of agreeing to a common set of scientific principles 
that all students must learn, faculty might agree to allow different scientific principles or 
content to satisfy the graduation or program requirements. Students could then be assessed 
on different scientific principles based on their major or program or, potentially, the courses 
in which they are enrolled. 

These alternative processes could be repeated in the other areas of General 
Education, but need not follow exactly the same path. In other words, there may be 
consensus on general principles in some areas (sciences, for example), but not necessarily in 
other areas such as arts and humanities. 

b. Skills 

The second area of assessing General Education focuses on skills, such as critical 
thinking, writing, speaking and listening, problem solving, quantitative reasoning or analysis 
and information competency. Since these General Education skills, sometimes referred to as 
“general intellectual skills,” are also taught and reinforced as part of the major, it is more 
likely that agreement on the need to assess them would be reached. There may also be 
consensus that students should develop these skills regardless of the program or courses 
selected. In California community colleges, for example, all college courses are expected to 
include development of critical thinking in their syllabi. Similarly, many colleges require 
competency in writing and mathematics for achieving a degree, with “information 
competency” likely to be added in the future. These skills offer opportunity for at least 
institutional agreement on definitions. 

Assessment of General Education 

In creating an effective assessment effort in General Education, it is suggested that 
the faculty begin with a discussion of content versus skills as briefly outlined above. One 
institution might decide to begin with the skills of General Education, especially since 
writing and some form and level of mathematics (or quantitative reasoning) are generally 
required for a degree, certificate and/or program completion. Likewise, critical thinking and 
information competency or literacy (library and computer technology) are also expected 
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competencies across institutions. A second institution might start with the General 
Education content required for a degree while a third might want to include some of both 
either at the beginning of assessment or as the effort develops. The subsequent areas of 
General Education, whether determined as skills or content, are called domains. Domains 
include the specific principles that students must learn and competencies that students must 
demonstrate. Appendix B provides an example of a definition of General Education 
developed by the Hawaii community college system. 

Regardless of how an institution defines General Education, a phase-in of assessment 
activities in this and other areas over time is likely to be more successful. Also, more likely 
to be successful is a cyclical approach to assessment, where parts of a total effort are carried 
out each year, making the assessment (and learning improvemenf) seamless and continuous. 



II. PROGRAM/ MAJOR 



Assessment in the Program and Major at two year colleges follows many of the same 
concepts and processes as assessment in General Education, although often considered to be 
somewhat easier to implement. This appears to be especially true for many vocational 
programs like nursing that have a rich tradition in program review. Such a program has 
more narrowly defined goals, objectives and outcomes, and often such outcomes-oriented 
assessment tools as a test for certification or licensing, a capstone course or experience, or 
similar method for assessing student learning as students exit the program. 

In developing an assessment effort with a Program or Major, the institution should 
begin by building on what is already available. The use of an effective program review 
process is very helpful and can serve as the platform for outcomes assessment. In this 
regard, the content and processes of program review can be combined with the type of 
processes used to assess student learning in General Education. Typical areas that an 
effective Program review would include are: 

• Industry trends 

• Program demand (community needs assessment) 

• Profile of both beginning and completing students (e.g., demographics, entering 
Basic Skills) 

• Student satisfaction 

• Student retention, completion and graduation 

• Graduate employment 

• Faculty assessment of the program 

• Employer satisfaction 

• Advisory committee feedback 

• Program efficiency (e.g., FTES/FTE or the number of full-time equivalent students 
divided by the number of full-time equivalent faculty) 

• External evaluator comments and recommendations 

**To this list would be added: Student Learning Outcomes. 
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In developing an effective assessment of Programs and Majors, the college might 
explore taking advantage of successful models already existing at the college. Many 
vocational programs, especially nursing, assess their outcomes and offer a good place to 
start. This can take the form of a modeling system, where the faculty from a particular area 
which has already developed, or can easily develop, a strong assessment system, offers a 
model of what is expected, included and implemented. This model will also include results 
and how they have been used. Then, the model is positively critiqued, improved by others 
and used to create similar models for other programs, tailored to the particular aspects of 
each program and major. The faculty from successful areas can be mentors for their 
colleagues in other areas as the college develops an institution-wide system. In this way, the 
college can develop an effective assessment program across all Programs and Majors with a 
phasing-in and cyclical process commonly employed with program review. 

Examples of Program Outcomes in several Majors are included in Appendix E. 
Examples of Program review processes that include outcomes assessment of student 
learning are presented in Appendix F. 



III. BASIC SKILLS 



One of the hallmarks of a comprehensive community college is the “open door 
policy” which translates into opportunity, often multiple opportunities, for a higher 
education for all adults who can benefit from such an education. “Who can benefit” is very 
broadly defined. With rare exceptions, people who are 1 8 or older, with or without a high 
school degree, are allowed to enroll at a community college. With such a wide range of 
backgrounds, experiences, skills, abilities and motivation, determining who can benefit from 
enrollment is difficult. 

Most community colleges choose to assess individual students at entry and use the 
results to place students in appropriate beginning courses. If the students succeed in 
achieving passing grades in their courses, they can continue toward their goal - whatever 
that is (e.g., skill improvement, certificate/degree, or transfer to a university). In practical 
terms, the ability to benefit from higher education (the open door policy) is typically 
determined by the student achieving satisfactory performance in courses. 

The open door policy permits access to higher education. Access without quality is 
empty at best and counterproductive at worst, by falsely raising expectations for a college 
education, only to be awarded a certificate or degree that is meaningless. Assessing the 
quality of education provided must be as important as focusing on access. At least equally 
important to the quality of what is offered, is the success of students in learning and 
achieving competency. This is the heart of student learning outcomes or SLO’s. 

Many students who come through the open door have basic skills of reading, writing 
and mathematics below, and often well below, the skills needed to handle college level 
courses. Given the underpreparedness of many entering students, an open door college 
(most two year, almost all community colleges and many four year institutions) must 
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address the skills of these entering students. Failure to do so will likely preclude achieving 
appropriate SLO’s in other areas such as general education and a major. 

Developmental education (sometimes referred to as basic skills or remedial 
programs) is a term used to describe a variety of services and courses designed to address 
the underprparedness of many entering students. (In California community colleges, the 
term “matriculation” is used to describe student services and processes, generally non- 
instructional, provided to assist entering students to successfully begin their college 
education.) A comprehensive developmental education program has five key components: 

1 . Placement Testing - should be mandatory, with exceptions such as prior testing 
and/or successful completion of prior coursework and includes, at a minimum, 
basic skills testing designed to determine the reading, writing and mathematics 
skills of entering students. 

2. Placement - refers to the enrollment in begirming courses on the basis of multiple 
variables including results of a placement test. 

3. Basic Skills Instruction - usually carried out through basic skills courses in 
reading, writing, ESL and mathematics. Because of the diversity of student 
underpreparedness, multi-level remedial/basic skills courses are almost always 
needed especially in community colleges. 

4. Support Services - include counseling/advising, a learning center, tutoring, 
supplemental instruction and similar programs geared to provide cognitive and 
affective support to students usually outside the classroom. 

5. Assessment — refers to the evaluation of the effectiveness, including outcomes, of 
the developmental education program. 

No open door college can be successful without an effective, comprehensive 
developmental education program. Given the low proficiency levels of many entering 
students in basic skills, a lack of success in developmental education will lead either to high 
attrition (empty access) or a lowering of standards (poor quality). Developmental education 
bridges the gap between the low skills levels of many entering students and the skills needed 
for success in college level courses. Thus, for open door institutions, assessment in 
developmental education is crucial. 

Basic Skills Instruction 

Instruction is the heart of developmental education and it is the heart of basic skills 
assessment. There are two main methods for assessing basic skills: direct assessment and 
indirect assessment. For writing, direct assessment includes having students demonstrate 
their level of competency or proficiency by writing; for math, it means having students solve 
problems and demonstrate that they understand the basic skills of mathematics. In general, 
reading is usually assessed by a reading test, but could also be assessed by oral readings or 
effective demonstration of understanding reading through writing about the material. While 
this latter method would make it difficult to separate reading from writing competency, most 
faculty would probably care less about such differentiation and be more concerned about 
whether a student could, in fact, demonstrate competency in both reading and writing. (The 
direct assessment of SLO’s in Basic Skills can differ little i n process from similar 
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assessment in General Education although the content of the writing and math skills is 
usually at a higher level in general education.) 

Assessment of Basic Skills, both direct and indirect, should be implemented within 
each discipline separately: reading, writing, English as a Second Language (ESL), and 
mathematics. For the purposes of this Handbook, assessment in ESL courses and programs 
should be considered analogous to the process for reading and writing. • • ' 

Direct Assessment of Basic Skills Assessment 

The crucial point of assessment of Basic Skills is to determine how well students 
have learned, that is, how proficient each student has become in writing, reading and 
mathematics. Key aspects include: 

1 . Define the reading, writing and math proficiencies expected to enter the college 
level English course (e.g., English 1 A) and the lowest (first) college level math 
course. These entering college level proficiencies are, or should be, the 
outcomes of the highest level Basic Skills courses (as well as the prerequisites for 
these same college courses). At a minimum, a college should assess the SLO’s 
of the highest level remedial courses offered at the college. 

2. Many colleges may choose to go beyond the outcomes of the highest level 
remedial courses and assess the skills of students as they complete lower levels 
of remedial courses. In this situation, the college should define the proficiencies 
(i.e., prerequisites) needed to enter each level of remedial course for each Basic 
Skills area. These proficiencies then become the outcomes of the lower level 
remedial course. (The lowest level course has no entering proficiencies or 
prerequisite which lead to certain difficulties discussed below.) 

3. Select the method used to assess proficiency in each Basic Skills area. 

Generally, this is a test, standardized or locally developed. Assessment in 
reading should generally be a standardized test due to the extraordinary difficulty 
of creating a locally developed reading test that is reliable and valid. Assessment 
in math (probably arithmetic and algebra) should focus on problem solving and 
be either locally developed or standardized. In writing, the most accurate 
measure of assessment includes both essay(s) and multiple choice questions, 
again either standardized or locally developed. (Essays can be standardized as 
well, as can be the scoring, such as the use of holistic scoring or, more recently, 
scoring using a computer program.) 

4. Decide whether to assess all students or a sample (usually randomly selected). In 
this decision, the institution needs to consider whether it wants all students to 
demonstrate proficiencies or whether program assessment is the focus. (The 
focus in Basic Skills assessment is more likely than other areas of student 
learning, to be on demonstration of competency for each student, rather than 
using a sampling procedure due to the need for these skills in higher courses, in 
most work situations and in most people’s lives.) 
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5. Finally, the location of the assessment must be decided including whether it is 
part of the course or a separate assessment activity. See the section below on 
methods and tools of assessment for a discussion of motivation in assessment and 
the use of embedded assessment techniques. 

Pre- and Post-Testing: While a pre-test is not essential in assessment, a pre- 
test/post-test design is more valuable in providing information about student learning, 
including value-added. Assessment (usually a test) is administered either at the beginning of 
the course or before the course begins; then an alternate form of the test is administered at 
the end of the course. The placement test can serve as a valid pre-test with an alternate form 
of the placement test used as the post-test. The simplest method is to compare the pre-test 
results to the post-test results with the traditional goal being to achieve a statistically 
significant increase from the pre- to the post-test. Of course, the comparison of results is 
only made with those students who complete both the pre- and the post-test. 

Finding positive results with this method is usually relatively easy to achieve for 
several reasons. First, the weakest students tend to drop out of the course and do not 
complete the post-test. This means the comparison more likely to be with the students who 
completed the course and, thus, it is expected, became more proficient. A second reason 
that positive results will be likely to ensue from this method is a phenomenon called 
“regression toward the mean”. This is a statistical situation that occurs when a measurement 
or assessment is made of one group that is at either end of the continuum being assessed. In 
this case, by definition, the students in the Basic Skills courses are below average in at least 
one basic skill. Everything else being equal, the probability is that some of these students 
will increase in proficiency (move or “regress” toward the mean) simply by chance. In other 
words, more of these students have only one direction to go: up. Therefore, since it should 
be relatively easy to achieve a statistically significant increase in pre- and post-testing, 
failure to do so would probably indicate serious problem(s) with the course or program. 

A second, more powerful pre/post-test design is to examine the percentage of 
students who both passed the Basic Skills course (C or better) and who achieved the 
standard of proficiency previously established to move to the next level of Basic Skills 
course or college level. With this latter method, then, standards for passing the course 
include demonstrating proficiency at the end of the course or program as well as 
achievement of a successful grade (C or higher). Using pre/post-testing in this way, the goal 
might be: 90% of all students achieving a “C” or better would be able to demonstrate 
successful proficiency on the post-test. This leaves room for up to 10% of the students who 
really have achieved proficiency even if they were not able to demonstrate such on the post- 
test due to extraneous problems such as test anxiety. This pre/nost-test design could be 
employed to assess learning, either “value-added” learning and/or competency in any course 
across the curriculum beyond Basic Skills. 

Warning: motivation is often a factor in post-test results, especially if students perceive that 
the post-test has little or no impact on their grade. Therefore, it is recommended that the 
post-test play an important role in the final grade, and, whenever feasible, be integrated into 
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the final exam process. The need to address student motivation is a crucial factor across the 
spectrum of assessment. See especially the section below on “course embedded” assessment 
and similar efforts to appropriately address student motivation issues. 

Indirect Assessment of Basic Skills 

While direct assessment of Basic Skills is essential to determine students’ 
competency, it is strongly recommended that indirect assessment also be used. The 
combirialioii of multiple variables produces a much more powerful assessment model 
(increases the probability of accuracy of results and conclusions) than any single variable, 
each of which, when used alone, may be flawed. Also, especially in the Basic Skills area, 
many variables play important roles in student learning and student success. How students 
perform in subsequent courses, the retention/persistence rates, and their overall grades 
combine to more accurately depict the success of the varied services that go into Basic Skills 
and remedial programs. Several indirect assessment variables should be considered (each of 
these could be assessed by semester or by year): 

a. Course Completion. Retention in California community colleges is defined as 
the percentage of students who are enrolled at the time of census who achieve a grade of A, 
B, or C in that course. Students who receive a D, F or W should be considered not 
successful. I’s (incompletes) are excluded from the analysis. The same comparison can be 
used for C (Credit) and NC (No Credit) grades. The flaw in using this variable in isolation 
is that it is not known under what standards grades are awarded or how much learning has 
actually taken place. 

b. Successful Performance in Subsequent Courses. This is defined as the 
percentage of students who, having successfully completed the Basic Skills course or 
program, subsequently enroll in and achieve a passing grade of “C” or better in the next 
level course of basic skill or college course. This would, obviously, be English to English 
and math to math. For reading, if there is no comparable college level course in reading, the 
college might select a General Education course where reading is an important prerequisite 
and which is taken by a relatively large cross section of students (e.g., psychology, history, 
political science, etc.). 

c. Persistence. This is the term used in California community colleges while the rest 
of the covmtry would probably use the term “retention”. This is defined as the percentage of 
students in the original cohort who continue to enroll at a subsequent point in time (e.g., one 
semester, a year, or more). Students who drop out and then return are continued in the 
cohort at the time the analysis or “snapshot” is done. The flaw in using this variable in 
isolation is to assume that students who persist have learned. 

d. Grade Point Average (GPA). The use of grade point average as one of the 
indirect measures of student learning provides a more accurate assessment than individual 
course grades because it includes the grades of multiple courses. Thus, GPA is more likely 
to account for individual differences inherent in each course separately. A simplistic 
definition would include the mean GPA achieved by each student either by semester or 
cumulatively. The college must also decide whether to include the Basic Skills or remedial 
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courses in the calculation of GPA. Another variation is to examine the percentage of 
students who achieve a minimum GPA such as 2.0 or 2.5. It is expected that students who 
have gained basic skills competencies at the college level would be better able to achieve a 
successful GPA since basic skills are, by definition, important prerequisites for most college 
courses. (The word “better” here leads to the question: against what? See Appendix A for a 
discussion of comparison groups.) 

e. Other Variables. Other possible indirect variables include credit ratio (the 
percent of credits or imits achieved compared with the credits or units attempted), 
satisfaction (student and/or faculty), transfer rates and graduation rates. Satisfaction Surveys 
can be very helpful in seeking information from those most directly involved in instruction 
about perceptions, processes, commimication, and so on, which frequently play important 
roles in achieving a successful program. Such surveys should be used in conjunction with 
the outcome variables described above, not replace them. Asking students how well they 
perceive their learning can be constructive, but is insufficient to demonstrating actual 
learning or competency. 

Transfer Rates and Graduation Rates are important indirect outcomes of educational 
programs and should be included in assessing program, degree and institutional outcomes. 
However, because of the numerous intervening variables, including time and other courses 
(e.g.. General Education, major), the relationship between Basic Skills efforts and transfer 
and graduation rates is rather tenuous. It is recommended that these latter two outcome 
variables be more appropriately used elsewhere in assessment than with Basic Skills. 

Academic Success is an indirect assessment variable that can help assess program 
success. Academic success is defined as the percentage of a cohort of students who 
remained enrolled after one, two or more semesters and who achieved a cumulative GPA of 
2.00 or higher at the time of the follow-up. This factor combines both course retention and 
satisfactory grades. 

How to Use These Variables in Assessing the Outcomes of Basic Skills Instruction 

Collecting information for assessment is relatively easy. Knowing how to interpret 
the data is much more difficult. Many factors enter into this process and analysis. 

“Success” is a complex phenomenon, neither easily defined nor understood. The emphasis 
of this Handbook is on developing effective practices of assessing outcomes. The multiple 
aspects successful learning, such as instruction, curriculum, standards, class size, time on 
task, student abilities and competencies, learning styles, instructional methodologies, and so 
on will not be thoroughly explored here. Faculty, however, must take these factors into 
consideration to best understand and interpret the data and use the results to improve 
learning. 

Many factors that enter into successful learning are also present in Basic Skills 
instruction. Remember that students take these Basic Skills courses not always out of choice 
but out of need. Therefore, motivation is frequently a factor; students find themselves 
enrolled in courses that are not their favorite, and perhaps, have not been successful in the 
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past. Also, almost by definition, students’ past poor academic performance in Basic Skills is 
related to such factors as: poor planning and time management, inadequate study skills,' 
learning disabilities, difficulties in delay of gratification, personal, social or family issues, 
and so on. While students in courses across the curriculum exhibit these factors, they 
frequently predominate among students enrolled in Basic Skills courses. An effective 
program or institution must take these factors into consideration, inside and outside the 
classroom. 

Another challenge of achieving success in Basic Skills is factoring in the diversity of 
skills, especially in the lowest level courses, where there is often no floor of skills. That is to 
say, some students could be illiterate, at least in English, in these courses. This significant 
diversity of students’ competencies in the lowest level Basic Skills course makes a standard 
notion of “success” more difficult to achieve. 

The diversity of Basic Skills above the lowest level course in a multi-level system of 
pre-college courses should be less than the lowest level course. This is because the higher 
level Basic Skills courses frequently have, or generally should have, established 
prerequisites that students must have to enter. The use of prerequisites establishes more 
reasonable levels of skills and competencies needed to learn the material in the course and 
increases the likelihood of success by both the instructor and the student. 

Comparison Groups 

One method of assessing outcomes is to examine whether and how well students 
have achieved competency, defined for a particular subject area or domain, by the faculty, at 
a predetermined level or standard. Another method of assessing success is to measure the 
growth or “value added” from the beginning of the course or program to the end. While 
providing important information, value added does not necessarily indicate competency, 
proficiency or achievement of a standard. A third method is to compare outcomes across 
different groups of students. This latter method is especially helpful in examining the 
success of one method, service or program against another. The expectation (or hypothesis) 
would be that those students served by the extra or special services or program would 
achieve better or higher outcomes, however defined. (For a more complete discussion of the 
use of control and comparison groups and suggested standards in Basic Skills assessment, 
please see Appendix A.) 

IV. STUDENT SERVICES/ACADEMIC SUPPORT SERVICES 

Astin (1991) of UCLA proposed a model of program evaluation that seems 
appropriate today in developing an assessment system for student services (counseling, 
financial aid, HOPS, DSPS, athletics, international education, PUENTE, TRIO, MESA, and 
so on) and academic support services (tutoring, learning center, supplemental instruction, 
etc.). Astin’s I-E-0 model looks at Input, Environment and Output. Input refers to what the 
students bring with them as they enter a program or institution: demographics, past 
academic achievement, basic skills proficiencies, learning styles, study skills, and various 
affective factors (e.g., motivation, etc.). The Environment includes the programs and 
services an institution provides for students including such components as: counseling. 
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instruction, matriculation, tutoring, mentoring, computer assisted learning, and so on. 

Output includes the outcomes/results of what has happened to the students who were served 
by the program/service/institution. 

Assessment of student services/academic support services could begin with Astin’s 
I-E-0 model of evaluation as a theoretical framework and then explore combining 
traditional program review processes with additional focus on the outcomes of the effort(s). 
Some or all of the following might be used in examining the process variables traditionally 
included in an effective program review: 

• Leadership 

• Lines of authority 

• Staffing 

- Number 

- Credentials 

• Number of Students served 

• Facilities 

• Budget 

• Climate 

• Satisfaction 

• Services offered 

An effective assessment design for student services/academic support services would 
add measurable outcomes to these traditional program review/process variables. As is true 
for the other areas, there are two major directions in assessing the efforts of Student 
Services/ Academic Support Services include: 

a. Direct assessment, and 

b. Indirect assessment of students served by a program or service (e.g., HOPS or 
counseling). 

The former is difficult to accomplish, especially in a two year college. The latter 
follows a pattern similar to effective program evaluation, with the addition of student 
outcomes. Each of these are: 

A. Direct Student Learning. This is an assessment program for student services 
similar in process and design to one for instructional programs. 

1 . Begin by defining the area of learning expected, including the definition of the 
learning outcomes expected in that area. Possible example areas include: self- 
esteem, self-confidence, leadership, etc. 

2. Define which students are expected to achieve the outcomes in the area(s) selected 
and the criteria for success. This becomes the cohort to be assessed, and may include 
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students in a single course or a program or all students enrolled at the college. The 
cohort might then be defined by time limits, units enrolled, etc. 

3. Select appropriate tools for assessment. A pre- and post-testing design might be 
employed to assess where the students are at the beginning of the process and where 
they are at the end of the process/course/service provided. Without assessing the 
students’ beginning level on a trait, it is not possible to determine the impact of the ' ' 
program or services on students’ leaming/development of that trait. The post-test 
assesses whether students have achieved the level expected on the trait(s) being 
assessed. The post-test can also provide information on the change (or “value 
added”) when compared to the pre-test. (A “test” is used here, especially with 
student services, to include a survey or other assessment tool.) 

4. Analyze the results of assessment. 

5. Use the results to improve a program/service and its outcomes. 

Caution: all of the complexities of experimental design, and the multitude of possible 
impacting variables, are beyond the scope of this Handbook. However, it is suggested that 
in assessing impact, do not get mired in debates over “cause and effect”. For example, 
suppose an institution wants to assess its impact on student self-esteem or leadership. Also, 
suppose the entering students (in a program or institution) generally score low on a pre-test 
of this trait. On the post-test, however, the students generally score high on this same trait. 

A reasonable person might conclude that the students improved or/achieved success on this 
trait because of what the program, service or college had done to improve student 
performance. Critics, on the other hand, might warn that other factors led to the 
changes/improvements found, such as maturation of the student or other factors or 
programs, on or off campus. While these critics are not wrong, educational 
programs/services do not permit the controlling of variables that pharmaceutical companies, 
for example, use to test the impact of drugs (so-called “double-blind” experiments where 
neither the patient nor the person who administers the drug knows whether an actual 
medicine or a placebo is given). 

We do not and cannot control all the possible variables (“intervening variables”) in 
education. The trap is to conclude that we cannot, therefore, make any reasonable 
conclusions. Not true! The use of multiple variables and the replication of studies are 
powerful tools to use in addressing these intervening issues. Other traditional methods such 
as random selection, use of control groups or. more likely, comparison groups, also can 
contribute significantly to addressing extraneous variables. 

B. Indirect Measures . In this assessment model, a cohort of students is identified and 
selected and then followed over different time periods. In this method, called longitudinal 
cohort analysis, it is important to emphasize that these cohorts do not change. A student 
may drop out of the program being assessed, or even out of the college, but will still remain 
in the cohort. If they re-enroll in the college or program, they continue to remain in the 
originally defined cohort. For example, a cohort of 100 students enters a program and, one 
year later, 62 of them are still enrolled in the program. The persistence rate would be 62% 
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regardless of the number of students who dropped out during the year only to reenroll prior 
to the beginning of the second year, when the follow-up is made. This procedure permits 
taking a picture or snapshot of what is and has been happening to the cohort at various time 
intervals on any of a variety of variables. 

An example of such a cohort would be the selection all EOPS students who began at 
the college during a particular semester. Let’s assume that the cohort consisted of 100 new 
EOPS students who began in the EOPS Program in the fall of 2000. We then take a picture 
of what has happened to those same 100 students one, two or more semesters or years later. 
We might look at how many and what percent of these students are still in the program, are 
still in the college, achieved a degree, transferred to a university, achieved a cumulative 
GPA of 2.0 or higher, and so on. This would give us feedback on what has happened to 
these students and can be used not only to assess the impact of the program, but also to 
examine the impact of the program on different cohorts, as well as provide information 
needed to improve the program. 

Please note that this kind longitudinal cohort analysis provides more valid 
information on the impact of a program on student outcomes and success than looking at the 
total student population of a program. In particular, sometimes a practice is followed where 
students who do not perform well in a program are replaced by new students. This practice 
changes the cohort and thus the assessment of the cohort/program. A change in the cohort in 
this way is very likely to give a distorted (and inaccurate) view of the outcomes of a 
program because it is not known whether the outcomes are a result of the program or the 
change in students (cohort). 

Outcome Variables 

Once the cohort is defined, the same indirect outcome variables used with other areas 
of learning can help assess how effective the program or service is over different time 
periods: 

• Course completion rates 

• Basic skills completion rates 

• Retention rates 

• Persistence rates 

• GPA (semester and cumulative) (mean or percent who achieve a certain level) 

• Graduation rates 

• Transfer rates 

• Success after transfer (GPA, Persistence, Graduation) 

• Job Placement rates 
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• Job/Employer satisfaction 

Comparisons using each of these variables can be made with each cohort and other 
cohorts in the same program, in different programs, or with the student population at large. 
These variables can also be used to make comparisons over time to see if program changes 
are actually demonstrating improvements in outcomes. 

Direct assessment of SLO’s for each area of student services could also be carried 
out. However, the impact of the student services program on the achievement of these 
SLO’s may well be marginal because they have not traditionally been responsible for 
achieving SLO’s in Basic Skills, General Education or Major. 



V. ACCESS AND EQUITY 



Two important aspects of program and institutional effectiveness are the extent to 
which a program or institution enrolls potential students and maintains a diversified student 
population (access) and the level of outcome achievement for various student groups 
(equity). 

Access 



Access has two components: the percentage of the community that is served by the 
college and the ratio of diversity in the student population as compared with the community. 
As a goal, a community college should serve a percentage of the community equal to at least 
the national or state averages. In addition, the percent of students enrolled for each of 
several defined groups (e.g., race/ethnicity, gender, etc.) should be reasonably comparable 
(within 5-10%) to the population where the college is located and community served. These 
comparisons can also be made over time to demonstrate changes both at the college level 
and in the community. 

Equity 



In the context of outcomes assessment, equity is defined as the extent to which an 
institution or program achieves a comparable level of outcomes, direct and indirect, for 
various groups of enrolled students. (These groups are most likely the same as defined in 
Access above.) In assessing equity, the methods of assessing outcomes are the same as 
those described in other areas of this Handbook . For example, after determining the 
graduation rate for all students in a particular cohort of entering students, assessing equity 
would mean assessing the graduation rates of various subgroups in the same cohort and 
seeing how these subgroups perform compared to the total cohort. This assessment process 
could be carried out for any cohort of students in any area of learning and for any program, 
degree, service or institution. 
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DEFINITIONS 



ACCESS: has two components: the percentage of the community that is served by the 
college and the ratio of mversity in the student population as compared with the community. 

ACCOUNTABILITY*: the public reporting of student, program or institutional data to 
justify decisions or policies. 

ANALYTICAL SCORING**: evaluating student work across multiple dimensions of 
performance rather than from an overall impression (holistic scoring). In analytic scoring, 
individual scores for each dimension are scored and reported. For example, analytic scoring 
of a history essay might include scores of the following dimensions: use of prior knowledge, 
application of principles, use of original source material to support a point of view, and 
composition. An overall impression of quality may be included in analytic scoring. 

ANCHOR(S) **: a sample of student work that exemplifies a specific level of performance. 
Raters use anchors to score student work, usually comparing the student performance to the 
anchor. For example, if student work was being scored on a scale of 1-5, there would 
typically be anchors (previously scored student work), exemplifying each point on the scale. 

ASSESSMENT: the systematic collection of data and information across courses, 
programs and the institution with a focus on outcomes, especially student learning 
outcomes, but also includes process, especially in seeking ongoing improvement. 

AUTHENTIC ASSESSMENT: assessment that requires students to perform a task rather 
than take a test in a real-life context or a context that simulates a real-life context. Designed 
to judge students' abilities to use specific knowledge and skills and actively demonstrate 
what they know rather than recognize or recall answers to questions. 

BASIC SKILLS: below college-level reading, writing, ESL, and mathematics. 

BENCHMARK: a sample of student work or a detailed description of a specific level of 
student performance that illustrates a category or score on a scoring rubric. 

CAPSTONE (course or experience): 

CLASSROOM ASSESSMENT TECHNIQUES (CAT's) 

COHORT: a group (of students). 

COMPETENCY*: a combination of skills, ability and knowledge needed to perform a 
specific task at a specified criterion. 

COURSE ASSESSMENT: assessment of student learning outcomes at the course level 
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CRITERIA**: guidelines, rules, characteristics, or dimensions that are used to judge the 
quality of student performance. Criteria indicate what we value in student responses, 
products or performances. They may be holistic, analytic, general, or specific. Scoring 
rubrics are based on criteria and define what the criteria mean and how they are used. 

CRITERION-REFERENCED ASSESSMENT**: an assessment where an individual's 
performance is compared to a specific learning objective or performance standard and not 
to the performance of other students. Criterion-referenced assessment tells us how well 
students are performing on specific goals or standards rather that just telling how their 
performance compares to a norm group of students nationally or locally. In criterion- 
referenced assessments, it is possible that none, or all, of the examinees will reach a 
particular goal or performance standard. 

DEVELOPMENTAL EDUCATION: a term used to describe basic skills/remedial 
courses and support systems (e.g., placement testing and placement, counseling/advising, 
and such academic support services as tutoring, learning center and computer-assisted 
instruction or CAI). (At COD: one of the academic divisions, focusing on adult basic 
education, non-credit ESL and GED (high school equivalency).) 

DIRECT ASSESSMENT: the measurement of actual student learning, competency or 
performance. Examples include essays, tests, speeches, recitals, capstone experiences and 
portfolios. 

DOMAIN: a set of skills or subskills in a particular educational area; for example, the 
specific skills that make up algebra or critical thinking. 

EMBEDDED ASSESSMENT: a method of sampling which allows broad assessment 
activities to be carried out within the course structure by “embedding” these activities within 
the course content, syllabus and assessment/grading practices, not separate from the course. 
This encourages students to be motivated and to perform to the best of their abilities. 

EQUITY: the extent to which an institution or program achieves a comparable level of 
outcomes, direct and indirect, for various groups of enrolled students. 

GENERAL EDUCATION: the content, skills and learning outcomes expected of students 
who achieve a college degree regardless of program or major. This includes both skills in 
such areas as writing, critical thinking, problem solving, quantitative reasoning, and 
information competency as well as content knowledge in a spectrum of learning outcomes 
including: communications, arts, humanities, mathematics, sciences and social sciences. 

HOLISTIC SCORING: a scoring process in which a score is based on an overall rating or 
judgment of a finished product compared to an agreed-upon standard for that task. 

ITEM**: an individual question or exercise in an assessment or evaluative instrument. 

INDIRECT ASSESSMENT: the measurement of variables that assume student learning 
such as retention/persistence, transfer and graduation rates, and surveys. 
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INPUT: the demographics and skills students bring with them as they enter a course, 
program or institution. 

INSTITUTIONAL EFFECTIVENESS: a term used by various components of the 
institution or the institution itself to review how effectively goals are achieved. 

ITEM**: an individual question or exercise in an assessment or evaluative instrument. 

LONGITUDINAL COHORT ANALYSIS: a form of evaluation or assessment where a 
particular group (cohort) is defined on a set of predetermined criteria and followed over time 
(longitudinal) on one or more variables. 

MATRICULATION: in California, a process to assist entering college students to be 
successful, including admissions, registration, orientation, placement testing, counseling, 
registration and evaluation. Outside of California: registration/enrollment. 

NORM-REFERENCED ASSESSMENT**: an assessment where student performance or 
performances are compared to a larger group. Usually the larger group or "norm group" is a 
national sample representing a wide and diverse cross-section of students. Students, schools, 
districts, and even states are compared or rank-ordered in relation to the norm group. The 
purpose of a norm-referenced assessment is usually to sort students and not to measure 
achievement towards some criterion of performance. 

OPEN-RESPONSE ITEMS: items requiring short written answers. 

OUTCOME: results; what is expected to be produced after certain services or processes. 
(See student learning outcomes below.) 

OUTPUT*: anything an institution or system produces 

• a value-neutral quantity measure 

• usually measured in terms of volume of work accomplished 

• often confused with a measure of quality of degrees, research, student services, etc. 
PERSISTENCE: the ongoing enrollment of students over multiple semesters or terms. 

PERFORMANCE-BASED ASSESSMENT: (also known as Authentic Assessment): 
items or tasks that require students to apply knowledge in real-world situations. 

PERFORMANCE INDICATORS*: a set of measures that are used to evaluate and report 
performance. 

PLACEMENT: the counseling/advising process, using multiple variables, usually 
including the results of a placement test, to assist entering college students enrolling in 
beginning college courses, especially remedial/basic skills courses. 

PLACEMENT TESTING: the process of assessing the basic skills proficiencies or 
competencies of entering college students. 
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PORTFOLIO: a representative collection of a student's work, including some evidence 
that the student has evaluated the quality of his or her own work. A method of evaluating the 
work is important as is determining the reasons the student has chosen the work included in 
the portfolio. 

PROGRAM ASSESSMENT: assessing the student learning outcomes or competencies of 
students in achieving a certificate/degree beyond basic skills and general education. 

PROGRAM REVIEW: a process of systematic evaluation of multiple variables of 
effectiveness and assessment of student learning outcomes of an instructional or student 
services program. 

PROMPT: a short statement or question that provides students a purpose for writing; also 
used in areas other than writing. 

PERFORMANCE INDICATORS*: a set of measures that are used to evaluate and report 
performance. 

RATER**: a person who evaluates or judges student performance on an assessment against 
specific criteria. 

RATER TRAINING**: the process of educating raters to evaluate student work and 
produce dependable scores. Typically, this process uses anchors to acquaint raters with 
criteria and scoring rubrics. Open discussions between raters and the trainer help to clarify 
scoring criteria and performance standards, and provide opportunities for raters to 
practice applying the rubric to student work. Rater training often includes an assessment of 
rater reliability that raters must pass in order to score actual student work. 

RELIABILITY**: the degree to which the results of an assessment are dependable and 
consistently measure particular student knowledge and/or skills. Reliability is an indication 
of the consistency of scores across raters, over time, or across different tasks or items that 
measure the same thing. Thus, reliability may be expressed as (a) the relationship between 
test items intended to measure the same skill or knowledge (item reliability), (b) the 
relationship between two administrations of the same test to the same student or students 
(test/retest reliability), or (c) the degree of agreement between two or more raters (rater 
reliability). An unreliable assessment cannot be valid. 

RETENTION: in California community colleges, the completion of a course or semester 
(Course Completion outside of California). Outside of California, used in the same manner 
as persistence: the reenrollment of students over multiple semesters or terms. 

RUBRIC: a rubric is a set of scoring guidelines for evaluating students' work. Typically a 

rubric will consist of a scale used to score students' work on a continuum of quality or 
mastery. Descriptors provide standards or criteria forjudging the work and assigning it to a 
particular place on the continuum. Rubrics make explicit the standards by which a student's 
work is to be judged and the criteria on which that judgment is based. (See Appendix D for 
more information on rubrics including two examples and several websites.) 
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SCAFFOLDING; giving support in order to help the performance of a task, whereby this 
support is faded. This contrasts with Modeling (to present a desired behavior or process so 
that it can be imitated by the learner) and Coaching (support to help the performance of a 
task aimed at improving the performance of the learner.) 

SCALE**: values given to student performance. Scales may be applied to individual items 
or performances, for example, checklists, i.e., yes or no; numerical, i.e., 1-6; or descriptive, 
i.e., the student presented multiple points of view to support her essay. Scaled scores occur 
when participants' responses to any number of items are combined and used to establish and 
place students on a single scale of performance. 

STANDARDIZATION**: a consistent set of procedures for designing, administering, and 
scoring an assessment. The purpose of standardization is to assure that all students are 
assessed under the same conditions so that their scores have the same meaning and are not 
influenced by differing conditions. Standardized procedures are very important when scores 
will be used to compare individuals or groups. 

STUDENT LEARNING OUTCOMES (SLO): the competencies and skills expected of 
students as they complete a course, program or institution. 

STANDARD*: a predetermined criterion of a level of student performance 

• a measure of competency 

• set by experts representing a variety of constituents (e.g., employers/ educators/ students/ 
community members) 

• criterion (standard) may be set within institution or externally by industry/ employers. 

TASK**: an activity, exercise, or question requiring students to solve a specific problem or 
demonstrate knowledge of specific topics or processes. 

VALIDITY**: the extent to which an assessment measures what it is supposed to measure 
and the extent to which inferences and actions made on the basis of test scores are 
appropriate and accurate. For example, if a student performs well on a reading test, how . 
confident are we that that student is a good reader? A valid standards-based assessment is 
aligned with the standards intended to be measured, provides an accurate and reliable 
estimate of students' performance relative to the standard, and is fair. An assessment cannot 
be valid if it is not reliable. 

VALUE ADDED*: a comparison of knowledge, skills, and developmental traits that 
students bring to the educational process with the knowledge, skills and developmental traits 
they demonstrate upon completion of the educational process. 

* Defined by the National Postsecondary Education Cooperative. A document that further 
examines issues related to the measurement and use of student outcomes and the complete 
dictionary of over 400 terms are available on the NPEC Web site (nces.ed.gov/npec). 

** Derived from the CRESST Glossary, Graduate School of Education, UCLA. 
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Websites 

There is an incredible amount of assessment, et al material on the web. If you use Google as 
a search engine, enter: “community college outcomes assessment”. This alone will keep you 
busy. Highlights include: 

www.ca-assessment-inst.org The website for the California Assessment Institutes. 

www.aahe.org/assessment This website from the American Association for Higher 
Education (AAHE) Assessment Forum provides a wide variety of excellent materials. 

http://nces.ed.gov./npec/evaltests/ Developed by the NPEC (National Postsecondary 
Education Cooperative), provides definitions of the domains of critical thinking, problem 
solving and writing as well as descriptions of tests used to assess these skills 

http://www2.acs.ncsu.edu/UPA/assmt/resource.htm An excellent array of internet resources 
for higher education outcomes assessment, NC State University, University Planning 
Analysis. 

http://ericae.net/testcol.htm The Test Locator at the Eric Clearinghouse on Assessment and 
Evaluation. Includes links to ETS database on tests and to reviews from Buros Mental 
Measurements Yearbooks and Test in Print. Guidelines for test selection tips are included to 
help in selecting appropriate instruments. 

www.calpress.com Publishes several instruments related to critical thinking as well as 
rubrics for holistically scoring essays. 

www.sonoma.edu/cthink/default.html Includes links on what critical thinking is and how to 
teach and assess it. 

www.Colorado.edu/outcomes/resource.html Internet resources for higher education 
outcomes assessment. University of Colorado at Boulder. 

http://www.tamu.edu/marshome/assess/oabooks.html A list of links of General Resources, 
Agencies, Institutes and Organizations, Assessment Instruments and Techniques, 
Assessment Papers and Reports, Commercial Resources on Assessment, Benchmarking, and 
Software. 

http://www.topsv.org/leameroutcomes.html — Links to assessment implementation, general 
education competencies, accreditation outcomes, and divisional competencies. 

http://www.cod.edU/outcomes/index.htm#DivisionProgram College of DuPage Program 

http://www.sinclair.edu/divisions/ — Sinclair Community College Academic Divisions page 
— Visit each division to view program outcomes. 
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http://www.ecc.edu/studentlife/student acad.php3 -- Erie Community College Academic 
Programs 

http://www.uncc.edu/stuaffairs/sar/ University of North Carolina at Charlotte Student 
Assessment Resources 

htt p://www.mcc.commnet.edu/irp/outcomes.htm — Manchester Community College 

http://www.nv.cc.va.us/assessment/index.htm . Academic Assessment at Northern Virginia 
Community College 

http://www.parldand.cc.il.us/aac/programs/natsci/index.htm Parkland College Academic 
Assessment 
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APPENDIX A 



The Use of Control/Comparison Groups and Standards in Basic Skills Assessment 



The use of control groups in assessing outcomes and effectiveness in higher 
education is difficult. We can’t control for “placebo” or “Hawthorne” (attention/change) 
effects, as is done in “double-blind” experiments used in drug research, where neither the 
patient nor the doctor or nurse know which persons are receiving the drug or the placebo. 
Both the students and the teachers know whether they are involved or not with the 
“treatment” (basic skills or remediation). 

Another difficulty in attempting to use a control group in education involves an 
ethical question. Is it appropriate to randomly select a group as a control, knowing that they 
need assistance, withhold that assistance and potentially jeopardize their college success? 

Fortunately, the use of “comparison groups” can work almost as well. (A thorough 
analysis of control and comparison groups, evaluation designs, statistics, and the impact of 
various factors on findings and interpretation of results is beyond the scope of this 
Handbook.) In the context of basic skills and remediation, a comparison group can be found 
which mimics control groups in several ways and permits appropriate analysis without 
causing ethical concerns. The following groups of students occur “naturally” in a 
community college and can be used as comparison groups to those students who enroll 
and/or complete basic skills and remediation. 

A. “No-Need” Group : these are entering students who do not need remediation in a 
particular Basic Skills area . This is an important comparison group. Please note, in 
carrying out appropriate assessment activities in Basic Skills, the definition of the 
group and the area of assessment should be defined by the subject area involved 
separately : reading, writing, and mathematics. To combine these Basic Skills areas, 
usually leads to confusion in interpreting the results and providing appropriate 
feedback as to which particular area is effective and which needs improvement. 

Also, to define comparison groups as “no need,” “needing remediation in one area,” 
“needing remediation in two areas” and “needing remediation in three areas” will 
most likely produce very predictable results: the more remediation needed by a 
student, the more likely performance will not be successful. This then, becomes an 
evaluation of the effect of multiple levels of remediation on student outcomes rather 
than as assessment of the effectiveness of the instruction involved. 

B. “Not Tested” Group. These are the entering students who should have been tested 
and assessed and were not. (It does not include transfer students and others not 
needing to be tested.) Please note: a successful placement testing and assessment 
program should essentially eliminate this group. 

C. “Not Placed” Group. These are students who need Basic Skills remediation, but do 
not enroll in the Basic Skills courses they need. An effective developmental 
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education program, including mandatory placement, would essentially eliminate this 
group. 

D. “Completed” Group. These are students who need remediation in an area (reading or 
writing or math), enrolled in the needed course or area and completed what was 
needed and achieved competency. This is the prime group for assessment in Basic 
Skills, because this group, more than any other, is expected to achieve the outcomes 
expected as a result of providing Basic Skills remediation and developmental 
education. It is the “treatment” group against which the other groups will be 
compared. 

E. “Not Completed” Group. These are the students who needed remediation but did not 
complete what was needed. This can be a comparison group, but by definition, is an 
unfair comparison. Students in this group are expected to perform poorly on the 
relevant variables or outcome area. Developmental education programs that cannot 
demonstrate sizeable improvement between the “Completed” group and the “Not 
Completed” group are not effective and not achieving acceptable outcomes. 



Comparisons 

In implementing an effective evaluation of Basic Skills, remedial and or 
developmental education programs, assessment should be able to demonstrate that the 
“Completed” group approximates the “No Need” group on a variety of variables including: 
proficiency (post-testing), retention and persistence, successful performance in subsequent 
courses, and grade point average. Multiple variables demonstrate that students whom the 
faculty assess as successful (the Completed group), complete the highest level course in 
each Basic Skill area (defined as receiving a “C” or better) and perform at a level 
comparable to those students who did not need remediation in that skill area (the No Need 
group). 

Standards for Assessing Basic Skills Instruction 

To place these concepts into a more quantitative format, the following standards may 
be utilized: 

Basic Skills Completion * - 70% of the students enrolled at census in the highest- 
level Basic Skills course should complete the course with a C or better. 

Post Test Competency - 90% of the students who achieve a C or better in the highest 
level Basic Skills course can demonstrate competency on the post-test where 
competency is defined at the level needed to meet the prerequisites for college level 
or defined as college level. (Conceptually, all students achieving a C or better, by 
definition, should have developed the competencies defined in the course. A 1 0% 
leeway is suggested to account for measurement errors in assessment, such as post- 
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test flaws or some students who are competent but not fully able to demonstrate it 
that day in that way.) 

Successful Performance in Subsequent Courses: Students in the “Completed” group 
(passed their highest level developmental course with a C or better) will receive 
grades in subsequent courses comparable to students who did not need . The concept 
here is essential to remediation: students who develop the competencies in previous 
remedial work are now adequately prepared to achieve academic success comparable 
to those students who did not need remediation in that subject area such as 
English/writing, reading, or mathematics. “Comparable” should be defined as a 
difference not greater than 10% between those not needing remediation (the “No 
Need” group, as determined by the initial assessment process for entering students) 
and the “Completed” group, defined as those who completed the Basic Skills 
course(s) in the particular skill area. (Warning; using mean grades may not be 
appropriate since, on average, students not needing remediation, by definition, had 
higher levels of competency and sometimes much higher level of competency, than 
that expected of developmental education students who complete their remedial 
courses.) 

Grade Point Average: As described above, this variable has many similarities, but is 
broader than a comparison of grades in only two courses. Consequently, the 
standard is similar: students who complete Basic Skills remediation should achieve 
GPA’s comparable to those not needing such assistance. Again, comparability 
should allow for a minimum disparity (less than 10% difference) and be based on 
percent achieving a 2.00 GPA or higher for reasons similar to those described for 
Performance in Subsequent Courses 

Persistence: In using this variable, the comparison would again be between those 
completing Basic Skills and those not needing it. Thus, the standard might be that 
students completing Basic Skills remediation would have a persistence rate 
comparable (within 10%) to those not needing remediation. 

It is again important to reinforce the use of multiple variables in developing 
an appropriate and successful assessment effort. The combination of these variables 
makes for a very powerful assessment methodology appropriately addressing the 
flaws inherent in using a single variable. 
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APPENDIX B 



A Model of General Education Outcomes 

(These were developed and adopted by the faculty of the community colleges of Hawaii.) 



GENERAL EDUCATION ACADEMIC SKILL STANDARDS 

The following academic skill standards for critical thinking, information retrieval 
and technology, oral communication, quantitative reasoning, and written 
commimication represent the minimum outcomes expected of students who have completed 
their General Education experiences. Each course included in the General Education 
curriculum should address at least one of these academic skill standards. 

Critical Thinking 

Critical thinking, an analytical and creative process, is essential to every content 
area and discipline. It is an integral part of information retrieval and 
technology, oral communication, quantitative reasoning, and written communication. 

Students should be able to: 

1 . Identify and state problems, issues, arguments, and questions contained in a body of 
information. 

2. Identify and analyze assumptions and underlying points of view relating to an issue 
or problem. 

3. Formulate research questions that require descriptive and explanatory analyses. 

4. Recognize and understand multiple modes of inquiry, including investigative 
methods based on observation and analysis. 

5. Evaluate a problem, distinguishing between relevant and irrelevant facts, opinions, 
assumptions, issues, values, and biases through the use of appropriate evidence. 

6. Apply problem-solving techniques and skills, including the rules of logic and logical 
sequence. 

7. Synthesize information from various sources, drawing appropriate conclusions. 

8. Communicate clearly and concisely the methods and results of logical reasoning. 

9. Reflect upon and evaluate their thought processes, value systems, and world views in 
comparison to those of others. 

Information Retrieval and Technology 

Information retrieval and technology are integral parts of every content area and 
discipline. 

Students should be able to: 
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1 . Use print and electronic information technology ethically and responsibly. 

2. Demonstrate knowledge of basic vocabulary, concepts, and operations of 
information retrieval and technology. 

3. Recognize, identify, and define an information need. 

4. Access and retrieve information through print and electronic media, evaluating the 
accuracy and authenticity of that information. 

5. Create, manage, organize, and communicate information through electronic media. 

6. Recognize changing technologies and make informed choices about their 
appropriateness and use. 

Oral Communication 

Oral communication is an integral part of every content area and discipline. 

Students should be able to: 

1 . Identify and analyze the audience and purpose of any intended communication. 

2. Gather, evaluate, select, and organize information for the communication. 

3. Use language, techniques, and strategies appropriate to the audience and occasion. 

4. Speak clearly and confidently, using the voice, volume, tone, and articulation 
appropriate to the audience and occasion. 

5. Summarize, analyze, and evaluate oral communications and ask coherent questions 
as needed. 

6. Use competent oral expression to initiate and sustain discussions. 

Quantitative Reasoning 

Quantitative reasoning can have applications in all content areas and disciplines. 

Students should be able to: 

1 . Apply numeric, graphic, and symbolic skills and other forms of quantitative 
reasoning accurately and appropriately. 

2. Demonstrate mastery of mathematical concepts, skills, and applications, using 
technology when appropriate. 

3. Communicate clearly and concisely the methods and results of quantitative problem 
solving. 

4. Formulate and test hypotheses using numerical experimentation. 

5. Define quantitative issues and problems, gather relevant information, analyze that 
information, and present results. 

6. Assess the validity of statistical conclusions. 

Written Communication 

Written communication is an integral part of every content area and discipline. 
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Students should be able to: 

1 . Use writing to discover and articulate ideas. 

2. Identify and analyze the audience and purpose for any intended communication. 

3. Choose language, style and organization appropriate to particular purposes and 
audiences. 

4. Gather information and document sources appropriately. 

5. Express a main idea as a thesis, hypothesis or other appropriate statement. 

6. Develop a main idea clearly and concisely with appropriate content. 

7. Demonstrate mastery of the conventions of writing, including grammar, spelling and 
mechanics. 

8. Demonstrate proficiency in revision and editing. 

9. Develop a personal voice in written communication. 
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APPENDIX C 



Examples of Standardized Tests for Assessing Student Learning Outcomes 

A. GENERAL EDUCATION 

1. Academic Profile (www.ets.org/hea/acpro') 

According to Educational Testing Service (ETS), the Academic Profile “focuses on 
skills developed in introductory courses in the humanities, social sciences and natural 
sciences. It concentrates on issues, themes and ideas. Questions do not ask for specific 
information but, instead, test a student’s ability to read carefully, make judgments about 
clarity, correctness, or organization of material, think critically about issues and arguments, 
and work effectively with mathematics. No specific courses or course patterns are 
assumed.” The Academic Profile consists of two parts: academic content and college skills. 

Academic Content 

Humanities: tests ability to read carefully passages on various humanities topics but does not 
measure knowledge in the humanities. 

Social Sciences: tests ability to think critically, etc., on subjects related to introductory social 
science classes, but does not measure knowledge of social science. 

Natural Sciences: tests ability to make judgments about argumentations, etc., within themes 
related to introductory natural science classes, but does not measure science knowledge. 

College Skills 

College Level Reading : norm-referenced and criterion-referenced subscores are provided. 
College Level Writing: norm-referenced and criterion-referenced subscores are provided. 
Using Mathematical Data : norm-referenced and criterion-referenced subscores are provided. 



2. Collegiate Assessment of General Education (CAAP) (www.act.org/caap) 

American College Testing (ACT) designed the CAAP to “assess academic 
achievement in selected general education skills that are considered critical for further 
academic work or for functioning well in society.” CAAP consists of five multiple choice 
sections plus an essay component, each of which can be administered alone or in any 
combination chosen by the institution. The six include: 

Writing Skills: measures understanding of standard written English including punctuation, 
grammar, sentence structure, and rhetorical skills such as strategy, organization and style. 
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Reading: measures reading comprehension by requiring students to read prose in fiction, 
humanities, social sciences and natural sciences and then reason, draw conclusions, and 
generalize beyond the written material. 

Mathematics: measures math skills in content areas ranging from pre-algebra to college 
algebra, emphasizing solution of quantitative problems and stressing applications and 
quantitative reasoning rather than memorization of formulas, knowledge of techniques, or 
computation skills. 

Critical Thinking: measures the ability to clarify, analyze, evaluate, and extend arguments 
using a variety of formats including case studies, debates, dialogues, overlapping positions, 
statistical argum^ents, experim.ental results and editorials. 

Science Reasoning: measures scientific reasoning skills based on principles taught in all 
lower-division science course rather than emphasizing factual knowledge. Students are 
required to interpret graphs, tables and scatter plots, analyze experimental results and 
compare alternative hypotheses and viewpoints. 

Writing/Essav: the only module not using multiple choice questions, measures students’ 
skills in formulating and supporting assertions about a given issue and in organizing and 
connecting major ideas. Requires two independent essays in response to two short passages. 



3. College BASE Test (C-BASE) (arc.missouri.edu/collegebase/text2.html) 

C-BASE is a criterion-referenced standardized achievement exam developed at the 
University of Missouri that assesses student proficiency in English, math, science and social 
studies. College BASE also measures cognitive processing skills in three cross-disciplinary 
competencies: interpretative reasoning, strategic reasoning and adaptive reasoning. 

Subject Areas 

English - 2 Clusters: Reading and Literature (reading comprehension and knowledge of 
major literary terms, genres, figures and works with emphasis on British and American 
literature) and Writing (pre-writing, composing, and revising skills using both multiple 
choice items and an essay). 

Math - 3 Clusters: General Math (computation, basic math concepts and statistical 
reasoning). Algebra (ability to solve linear equations, inequalities and quadratic equations), 
and Geometry (basic geometrical concepts and ability to use these concepts in calculations). 

Science - 2 Clusters: Laboratory/Field Work (applications of the scientific method) and 
Fundamental Concepts (knowledge of the basic principles of life, the earth and the physical 
sciences). 




61 



57 



Handbook on Outcomes Assessment 



Edward A. Morante 



Social Science - 2 Clusters: History (knowledge of chronology, historical movements, 
figures and institutions and causal relationships) and the Social Sciences (knowledge of the 
principal concepts of geography, economics and political science). 



B. VOCATIONAL MAJOR/PROGRAM . 

In contrast to the tests listed above, the National Occupational Competency Testing 
Institute (NOCTI) ('www.nocti.org) is a testing company that develops tests and provides 
related services to educational institutions and companies. Currently, over 170 tests are 
available in a variety of vocational/occupational fields including: business, computers, 
construction trades, consumer economics, culinary arts, drafting, electrical/electronics, 
heating/air conditioning, machine trades, maintenance services, media services, and 
transportation/small engine. 
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APPENDIX D 

SCORING RUBRICS 



Rubrics are guidelines for scoring when using performance or authentic assessment. 



Example One: The following rubric for scoring essays was developed by the faculty at 
Kauai Community College in Hawaii. This provides not only a useful scoring rubric for 
writing, but could also serve as a model for creating scoring rubrics in other academic areas. 



A (4 points) 


B (3 points) 


C (2 points) 


D (1 point) 


Clearly & effectively 
responds to 
assignment. 


Response to 
assignment generally 
adequate & thorough. 


Minimally responds to 
the assignment. 


Does not respond well 
to assignment. 


Demonstrates specific 
attention to 
relationship between 
audience & purpose. 


Demonstrates 
understanding of 
audience & purpose. 


Demonstrates some 
understanding of 
audience & purpose. 


Demonstrates poor 
understanding of 
audience & essay 
purpose. 


Main idea (thesis) 
very clearly stated & 
topic is effectively 
limited. 


Main idea clear & 
topic is limited. 


Main idea clear or 
implicit & topic is 
partially limited. 


Main idea unclear & 
topic only partially 
limited. 


Thesis supported in 
body of paper by a 
variety of relevant 
facts, examples, & 
illustrations from 
experience, references 
to related readings, 
etc. 


Thesis well supported 
in body of paper by 
facts, examples, 
illustrations though 
support may not be as 
vivid as the “A” essay. 


Thesis generally 
supported in body of 
paper by facts, 
examples, details. No 
more than one 
paragraph with 
inadequate support. 


Thesis supported in 
body of paper by few 
facts, examples, 
details. More than one 
paragraph with 
inadequate support. 


Organization & 
structure very evident: 
major points divided 
into paragraphs and 
signaled by use of 
transitions. Each 
paragraph has a topic 
sentence; sentences 
within each paragraph 
relate to each other & 
are subordinate to the 
topic. Introduction & 
conclusion effectively 
related to the whole. 


Organization & 
structure clear. Most 
major points are 
separated into 
paragraphs and 
signaled by 

transitions. Paragraphs 
are built on related 
sentences that 
logically develop the 
main points. No major 
digressions. 
Introduction & 
conclusion effectively 
related to the whole. 


Organization & 
structure mostly clear. 
Many major points are 
separated into 
paragraphs and 
signaled by 
transitions. Most 
points are logically 
developed. There may 
be a few minor 
digressions but no 
major ones. 
Introduction & 
conclusion are 
somewhat effective. 


The organization & 
structure must be 
inferred by the reader. 
Only some major 
points are set off by 
paragraphs and are 
signaled by 
transitions. There are 
some logically 
coimected points. 
There may be some 
major digressions. 
Introduction and 
conclusion may be 
lacking or ineffective. 
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Voice & tone are 
consistent & 
appropriate to the 
audience/purpose. 


Voice & tone 
consistent & 
appropriate although 
somewhat generic or 
predictable in places. 


Voice & tone 
adequate to 
audience/purpose 
although often generic 
or predictable. 


Voice noticeably 
generic or 
inappropriate (e.g. 
first person narrative 
may predominate in 
an analysis 
assignment). Tone is 
often inappropriate. 


Full variety of 
sentence structures 
used correctly. Word 
choice interesting, 
accurate and 
contributes to the 
writer’s ability to 
communicate the 
purpose. 


Variety of sentence 
structures used 
correctly despite an 
occasional flaw. 
Accurate &b varied 
word choice. 


Sentences & word 
choice predictable. 
Occasional errors in 
sentence structure, 
usage & mechanics do 
not interfere with 
writer’s ability to 
communicate the 
purpose. 


Little sentence 
structure variety; 
wording predictable; 
few synonym 
alternatives used. 
Errors in sentence 
structure, usage &B 
mechanics sometimes 
interfere with the 
writer’s ability to 
communicate the 
purpose. 


Few, if any, minor 
errors in sentence 
construction, usage, 
grammar, or 
mechanics. 


There may be a few 
minor or major errors 
in sentence 
construction, usage, 
grammar, or 
mechanics. 


There are some 
common errors (major 
and minor) in sentence 
construction and 
mechanics but the 
writer generally 
demonstrates a correct 
sense of syntax. 


There are numerous 
minor errors and some 
major errors. Sentence 
construction is below 
mastery and may 
display a pattern of 
errors in usage and 
mechanics. 


Source material is 
incorporated logically 
& insightfully. 
Sources are 
documented 
accurately. 


Source material 
incorporated logically. 
Sources documented 
accurately. 


Source material 
incorporated 
adequately & usually 
documented 
accurately. 


Source material 
incorporated but 
sometimes 
inappropriately or 
unclearly. 
Documentation is 
accurate only 
occasionally. 



Example Two : Tasks in Critical Thinking (ETS) Scoring Rubrics. This was originally 
developed by New Jersey college faculty as part of a statewide assessment effort. 

Core scoring method: Analysis & Inquiry 
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1 . Not proficient: A response was attempted but students scoring at this level either did 
not understand the questions or their explanations were erroneous, illogical, totally 
unrelated to the requirements. 

2. Limited proficiency: The basic requirements were not met, and responses were very 
brief, inappropriate, and/or incorrect. Responses were vaguely expressed or 
inaccurate. 

3 . Some proficiency: Student understood the question, yet the basic requirements were 
not met. Responses were vague, incomplete, and/or inappropriate. 

4. Fully proficient: The Core Score means that the questions were understood and the 
responses were correct and complete. Students met all basic requirements. 

5. Exceeds requirements: Students met all the basic requirements and provided some 
expansion or extension — citing evidence, providing additional information, or in 
some other way going beyond what was required. 

6. Superior performance: All basic requirements were met and expanded upon; in 
addition students presented ideas, interpretations, relationships, or examples that 
showed originality and insight. 

Holistic: Communication 

1. Not proficient: A paper demonstrating incompetence. It is seriously flawed by very 
poor organization, very thin development, and/or usage and syntactic errors so severe 
that meaning is somewhat obscured. 

2. Limited proficiency: A paper flawed by weaknesses such as failure to develop the 
required assignment, poor organization, thin development, using little or 
inappropriate detail to support ideas, and/or displaying frequent errors in grammar, 
diction, and sentence structure. 

3. Some proficiency: A slightly less than adequate paper that addresses the writing 
task in a vague or unclear way, shows inadequate organization or development, 
and/or has an accumulation of errors in grammar, diction, or sentence structure. 

4. Fully proficient: This is an adequate paper with only occasional errors or lapses 

in quality. It is organized, somewhat developed and uses examples to support ideas. 

It shows a basic command of, and adequate facility in, use of language. 

5. Exceeds requirements: A very strong paper vvdth only occasional errors or lapses in 
quality. It is generally well organized and developed, displaying facility in language, 
range of vocabulary and some variety in sentence structure. 

6. Superior performance: A superior paper that is well organized and developed, using 
appropriate examples to support ideas. It displays facility in language, range of 
vocabulary, and variety in sentence structure. 

OT Off topic, this designation is for responses that were completely off the assigned 

topic. 

Omit No response was attempted. 
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More information about rubrics can be found on the following websites. 

http://www. wcer. wise, edu/nise/cll/ilas/ 
http://edweb. sdsu. edu/webguest/rubrics/weblessons. him 
http://www. siue. edu/~deder/assess/index. htmU 

http://school. discovery, com/schrocksuide/assess. htmU Desimed for K-12 education, but 
useful as a model and adaptable for higher education performance assessments) 
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APPENDIX E 

Examples of Two Year College Program Outcomes 

I. Vocational Nursing Program Outcomes (Kingwood College) 

1. Provide nursing care for clients in structured health care settings who are experiencing 
common, well-defined health problems with predictable outcomes. 

a. Assist in the determination of the health status and health needs of clients 

b. Assist in the formulation of goals and a plan of care for the client 

c. Implement plan of care within legal and ethical parameters 

d. Support the implementation of nursing care by applying a working knowledge of 
client's rights, protecting the rights and dignity of the client and respecting the 
rights of others to have their own value system. 

e. Assist in the evaluation of the individual client's responses to nursing 
interventions 

f. Use the problem-solving approach as the basis for decision making in practice 

g. Provide direct basic care to assign multiple clients in structured settings. 

2. Utilize effective communication techniques with clients, families, and health 
care team members. 

a. Communicate significant findings to the health team 

b. Recognize and communicate ethical and legal concerns 

3. Contribute to the development and implementation of teaching plan for client 
with common health care problems. 

4. Assist in the coordination of human and material resources for the provision of 
care to assigned clients. 

a. Participate in discussion relating to the evaluation of client care with health team. 

b. Aid in identifying others who can assist in client care. 

c. Participate in the identification of client needs for referral to appropriate sources 
of assistance 

d. Participate in activities that support the organizational framework of structured 
health care settings 

e. Implement established cost containment measures in direct client care. 




67 



63 



Handbook on Outcomes Assessment 



Edward A. Morante 



II. Accounting AAS 

Program Objectives 

Graduates in the A.A.S. Accounting Program should: 

• Have the ability to analyze and prepare basic journal entries. 

• Be familiar with the various types and classifications of general ledger accounts used 
by the typical business activity. 

• Possess general computer competence with the ability to input computer data. 

• Be able to perform the various steps in the accounting cycle, including the 
preparation of basic financial statements. 

• Have the skills to do a variety of computations and apply logic as needed. 

• Possess the ability to apply specific Generally Accepted Accounting Principles in a 
variety of business situations. 

• Have general knowledge of Federal taxation rules for individuals, the differences 
between various taxable entities such as individuals, corporations and partnerships, 
and the means and techniques of tax planning and preparation. 

• Possess general knowledge of budgeting and product costing techniques and 
methods related to the control and evaluation of business operations. 

• Be familiar with auditing standards and responsibilities, types of audit reports, and 
auditing procedures and techniques. 

• Be aware of and able to use technical and professional journals and publications. 

• Possess the ability to communicate effectively in English, both in writing and in 
speaking. 

III. EARLY CHILDHOOD 

Program Goals: 

1. 0 To provide a comprehensive educational program for people working with young 

children in child care settings. 
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1 . 1 Students choose among the AAS degree, a Certificate for the Child 
Development Assistant, and a Career Studies Certificate in School Age Child Care; 
programs are inter-related and mutually supportive. 

2.0 To provide for the development of a knowledge base for understanding child 

development, and the acquisition of skills for preparing and maintaining appropriate 
programs for children. • ■ 

2.1 Students will demonstrate knowledge in all areas of child development: 
physical growth and development, language development, emotional and social 
development, and cognitive development of children ages 0-12. 

2.2 Students will demonstrate skills in working with children who have 
exceptionalities and/or developmental delays. 

2.3 Students will demonstrate knowledge of curriculum development and 
implementation, supportive of children from ages 0-12. 

3.0 To imbed all aspects of the program with a multi-cultural perspective, preparing and 
supporting students to work with diverse populations, in diverse settings, with children 0 - 
12 . 



3.1 Students will demonstrate knowledge and skills in working with diverse groups 
including children and their families, as well as fellow students and members of the 
professional community. 

4.0 To support dispositions for learning, provide for the attainment of general education 
goals, and encourage a commitment to life long learning. 

4. 1 Students will write clearly and in a professional manner, for a variety of 
purposes such as: curriculum planning, observation reports, research papers. 

4.2 Students will read and comprehend materials such as texts, journal articles, web 
resources, newspaper articles. 

4.3 Students will use technology to communicate with instructors and each other; 
explore resources available through the internet; and become knowledgeable about issues 
relating to technology and young children. 

5.0 To support professional development and credentialing through recognized programs 
and organizations. 

5.1 Students will demonstrate knowledge of national, state and local resources 
relating to child and family issues, including health and safety, legal rights of families, and 
intervention and support agencies. 
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6.0 To provide quality experiences in professional development, for renewal of professional 
credentials such as CDA and teacher certification, or to fulfill personal goals. 

6. 1 The program is responsive to community organizations, providing a variety of 
training and educational needs. 



IV. Music Program 

Program Goals and Objectives 

Students who plan to obtain an A. A. A. degree, to earn a Certificate in Music Recording 

Technology, or to transfer to a four- year institution to complete the Baccalaureate Degree in 

Music, Music Education, Performance, or Composition must develop specific competencies. 

Students taking music as an elective or for enrichment may achieve these competencies. The 

competencies are embodied in the following music program goals and objectives: 

1 . Students will develop the ability to make music of various styles and cultures by 
performances, class directed studies, and creative improvisation. 

• All students will develop ensemble skills (balance, blend, intonation, style, 
precision) by performing in various ensembles such as chorus, madrigal singers, 
concert band, jazz ensembles, and orchestra. 

• Instrumental and keyboard majors will perform major and minor scales and 
arpeggios as well as perform standard Grade IV (or higher) literature demonstrating 
good tone, intonation, articulation and musicianship. 

• Vocalists will perform standard repertoire in various languages demonstrating good 
tone, intonation, style, diction and musicianship. 

• Jazz students, on various levels, will be able to demonstrate improvisational skills, 
harmonic usage applicable to instrumental and vocal performance, and employ 
knowledge of structure/form and stylistic distinctions idiomatic to the genre. 

• Keyboard and theory students will improvise simple accompaniments to traditional 
songs using basic chord progressions, execute rhythms in simple and compound 
meters, and sight sing notation at various degrees of difficulty. 

2. Students will have a working knowledge of Western and World Music: its history, 
literature, forms and relationship to other arts, philosophies and sciences. 

Students will 

• recognize the characteristics of various styles in music (e.g. Medieval, Renaissance, 
Baroque, Classical, Romantic and Twentieth Century eras). 
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• be familiar with major composers and their works ("B minor Mass" of Johann 
Sebastian Bach, "Le Sacre de Printemps" of Igor Stravinsky and "Concerto for 
Cootie" of Duke Ellington.) 

• relate music to parallel movements in art, science, philosophy, literature and history. 

• identify and describe various forms, idioms and compositional devices used in 
music; (i.e. ground bass, rondo form, sonata, symphony, concerto, etc.). 

3. Students will demonstrate an ability to listen to music both analytically and 
critically. 

Students will be able to 

• identify monophonic, polyphonic and homophonic textures; Forms (phrases, periods, 
ABA, AB, sonata, rondo, theme and variations, minuet and trio); duple, triple and 
quadruple simple and compound meters; and Medieval, Renaissance, Baroque, 
Classical, Romantic and Twentieth Century styles. 

• make aesthetic judgments based upon their critical listening experiences 

4. Students will be able to use the language of music; its notational system, 
terminology, harmony, melody, rhythm and form, for the interpretation, 
creation and analysis of music. 

Students will be able to 

• sing, play, and write major and minor scales as well as major, minor, diminished and 
augmented triads 

• sing, analyze and harmonize a Bach chorale styled melody containing modulations 
and non- harmonic tones 

• analyze and utilize music from the common practice (1 8th century era) 

• identify and utilize major trends found in twentieth century compositional techniques 

• perform rhythms in both simple and compound meters. 

• transcribe into notation melodies, harmonies and rhythms played for them 

• demonstrate basic musicianship skills at the keyboard 

Students of jazz will be able to relate all of the above objectives to the jazz/pop idiom. 



5. Students enrolled in Music Recording Technology courses will have 
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knowledge of an experience using the recording equipment and theory necessary to 
produce a professional quality recording. 

Students will: 

• have experience using pitch transposers, 8 and 16 channel recording/reproducing 
systems; parametric and graphic equalizers, gaited compressors and limiters, time 
codes and digital effects devices. 

• be able to demonstrate the use of multi- machine sync recording using time code 
tracks, all elements of the operation of 8 and 16 channel recorder, reproducer 
systems and the operation and use of audio oscillators, sine and square wave 
generators, oscilloscopes, sound pressure measuring devices, and real time analyzers. 

• be able to demonstrate a knowledge of the theory and application of basic electronic 
theory, cable construction/ fabrication, machine calibration procedures (including 
head alignment, biasing and equalization), mechanical and time code editing as well 
as studio design (including acoustic considerations, sound wave theory and material 
selections). 

Through practical laboratory experience students will utilize their knowledge by preparing 
and creating master tapes using all of the sound processing equipment, microphone and 
recorder transports available. 
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APPENDIX F 



Examples of PROGRAM REVIEW 
I. Northern Virginia Community College 

Northern Virginia Community College determines that their academic programs are 
effective and that our students are learning by assessing the degree to which: 

• students are learning the knowledge, skills, and habits of thought necessary to 
achieve the program/discipline goals and objectives 

• the program/discipline goals derive from and support the college mission and goals, 
the general education goals, and the purpose of the program/discipline 

• the curriculum is coherent, current and consistent 

• the instruction is effective in enabling student learning 

• the resources are adequate for the production an environment conducive to student 
learning 

Additionally, through the inclusion of action plans, program/discipline reviews demonstrate 
that assessment results will be used in the improvement of student learning within the 
program/discipline. Finally, program/disciplines reviews provide information essential to 
effective plarming and budgeting as well as to process of evaluating our effectiveness as an 
institution. 

I. Student Learning Outcomes 

• Student competencies are consistent with the purpose of the program. 

• Students are achieving the goals and objectives of the program/discipline. 

• Students are achieving the general education goals of the college. 

• Student accomplishment is assessed through multiple means. 

• Assessment results are used to improve student learning. 

II. Program and Discipline Goals 

• Program/discipline goals are derived from and support the mission and goals of the 
college. 

• Program/discipline goals support the general education goals of the college. 

• Program/discipline goals are consistent with expectations of businesses employing 
students, transfer institutions receiving students, and the needs of the community 
served. 

• Program goals are consistent with the purpose of the program as stated in the 
catalogue. 

• Discipline goals make explicit the contribution of the discipline to student 
achievement of the general education goals, the goals of programs of which it is a 
significant part, and/or other core competencies required of graduates. 
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III. Curriculum 



• Program/discipline course content is consistent with the best practices in the field 
and the current thinking in the discipline. 

• Program/discipline courses display coherence through appropriate sequencing. 

• Program/discipline course content is consistent across the college and across various 
instructional delivery systems. 

• The curriculum of the program is comprehensive in addressing program goals, 
general education goals, and other core competencies required by the college. 

• The catalog accurately reflects the purpose and the curriculum of the 
program/discipline. 

• The curriculum of the program/discipline is monitored through the updating of 
course content summaries. 

• Information is sought from employers, graduates, advisory committees, and transfer 
institutions to assure the currency of the curriculum. 

IV. Instruction 



• Students experience appropriate methods of instruction that meet their learning 
needs. 

• Instructional modalities are consistent with the best practices in pedagogy and 
current thinking in learning theory. 

• Instructional modalities are demonstrably effective in producing student learning. 

• Instructional modalities are appropriate to the purpose and goals of the 
program/discipline. 

• Studs have access to various distance learning opportunities and instructional 
delivery systems such as web-based courses, computer-based courses, and courses 
delivered through the Extended Learning Institute. 

V. Resources 



• Fiscal resources are adequate to maintain and improve the program. 

• Instructional support is available, adequate, accessible, and appropriate. 

• Classroom and laboratory space is appropriate and adequate. 

• Faculty development experiences are appropriate, available, and adequate. 

• Faculty participate in improvement/development activities. 

• Students express satisfaction with the physical environment such as classroom and 
labs. 

• Students express satisfaction vsdth the accessibility of appropriate learning assistance 
services and resources. 

• Counselors work with faculty to ensure appropriate advisement. 

• The materials accessible through the library that support learning are current, 
available and the library staff is helpful. 
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II. Kirkwood College 

Program Outcomes: Examples of Evaluation 
Questions. Variables, and Data Sources 

Evaluation Questions Variables Possible Data 

Sources 



A. Program Completion 

1. How many students/trainees completed 
the training program? 

2. To what extent were students' objectives 
met? What is degree of completers' 
satisfaction with training program? 

3. To what extent did students increase 
their literacy skills? 

4. To what extent did students/trainees 
increase their vocational skills and 
knowledge? 



5. How many completers were placed in 
jobs following training? How many were 
placed in full-time and part-time 
training-related jobs? 

6. What is the average gross wage per hour 
of completers who were placed in jobs? 

7. What is the average number of hours 
worked per week of those placed in 
jobs? 

8. How many completers were placed in or 
enrolled in further education or training 
programs following completion of 
training? 

B. FoIIow-Up 

1. How many program completers were 
employed six months following initial 
placement in training-related and non- 
training related, Full-time and part-time 
jobs? 



Number of Completions 


Program 

Records 


Ratings of Satisfaction 


Completers 


Pre/Post Measures of 
Literacy Skills; Achievement 
of Training Objectives 


Program 

Records 


Achievement of Vocational 
Training Objectives and 
Competencies; Pre/Post 
Measures of Vocational 
Skills and Knowledge 


Completers; 

Program 

Records 


Number of Completers 
Placed in Jobs 


Completers; 

Program 

Reports 


Wages Per Horn- 


Completers 


Number of Hours Worked 


Completers 


Number of Completers 
Placed 

or Enrolled for Further 
Education or Training. 


Completers 


Number of Completers 
Employed 


Completers; 

Employers 
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2. What is average gross wage per hour of 
completers six months following initial 
placement? 

3. What is the average number of hours 
worked per week of completers six 
months following initial placement? 

4. How many completers are enrolled in 
school or other training program six 
months following initial placement? 

5. How many completers are unemployed 
six months following initial placement? 

6. How many completers were employed at 
any time during the six month period 
following initial placement? 

7. How many completers obtained job 
promotions during the six month period 
following initial placement? 

8. How many completers received wage 
increases or obtained a higher paying job 
during the six month period following 
initial placement? 

9. How satisfied are former participants 
with the jobs six months following 
initial placement? 

10. How satisfied are employers with the 
vocational and literacy skills of the 
program completers they hired? 



Wages Per Hour 


Completers; 

Employers 


Number of Hours Worked 


Completers; 

Employers 


Number of Completers 
Enrolled in School or 
Training Program 


Completers 


Number of Completers 
Unemployed 


Completers 


Number of Completers 
Employed at Any Time 
During Follow-up Period 


Completers; 

Employers 


Number of Completers Who 
Obtained Job Promotions or 
Higher Level jobs 


Completers; 

Employers 


Number of Completers Who 
Received Wage Increase 


Completers; 

Employers 


Ratings of Job Satisfaction 


Completers 


Ratings of Employee 
Satisfaction with Vocational 
Skills and English Language 
Skills 


Employers 
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APPENDIX G 

9 Principles of Good Practice for Assessing Student Learning 

American Association for Higher Education ASSESSMENT FORUM 

1 . The assessment of student learning begins with educational values. Assessment is not an 
end in itself but a vehicle for educational improvement. Its effective practice, then, begins 
with and enacts a vision of the kinds of learning we most value for students and strive to 
help them achieve. Educational values should drive not only what we choose to assess but 
also how we do so. Where questions about educational mission and values are skipped over, 
assessment threatens to be an exercise in measuring whaf s easy, rather than a process of 
improving what we really care about. 

2. Assessment is most effective when it reflects an understanding of learning as 
multidimensional, integrated, and revealed in performance over time. Learning is a 
complex process. It entails not only what students know but what they can do with what they 
know; it involves not only knowledge and abilities but values, attitudes, and habits of mind 
that affect both academic success and performance beyond the classroom. Assessment 
should reflect these understandings by employing a diverse array of methods, including 
those that call for actual performance, using them over time so as to reveal change, growth, 
and increasing degrees of integration. Such an approach aims for a more complete and 
accurate picture of learning, and therefore firmer bases for improving our students' 
educational experience. 

3. Assessment works best when the programs it seeks to improve have clear, explicitly 
stated purposes. Assessment is a goal-oriented process. It entails comparing educational 
performance with educational purposes and expectations — those derived from the 
institution's mission, from faculty intentions in program and course design, and from 
knowledge of students' own goals. Where program purposes lack specificity or agreement, 
assessment as a process pushes a campus toward clarity about where to aim and what 
standards to apply; assessment also prompts attention to where and how program goals will 
be taught and learned. Clear, shared, implementable goals are the cornerstone for assessment 
that is focused and useful. 

4. Assessment requires attention to outcomes but also and equally to the experiences that 
lead to those outcomes. Information about outcomes is of high importance; where students 
"end up" matters greatly. But to improve outcomes, we need to know about student 
experience along the way — about the curricula, teaching, and kind of student effort that lead 
to particular outcomes. Assessment can help us understand which students learn best under 
what conditions; with such knowledge comes the capacity to improve the whole of their 
learning. 

5. Assessment works best when it is ongoing not episodic. Assessment is a process whose 
power is cumulative. Though isolated, "one-shot" assessment can be better than none, 
improvement is best fostered when assessment entails a linked series of activities undertaken 
over time. This may mean tracking the process of individual students, or of cohorts of 
students; it may mean collecting the same examples of student performance or using the 
same instrument semester after semester. The point is to monitor progress toward intended 
goals in a spirit of continuous improvement. Along the way, the assessment process itself 
should be evaluated and refined in light of emerging insights. 

6. Assessment fosters wider improvement when representatives from across the 
educational community are involved. Student learning is a campus-wide responsibility, 
and assessment is a way of enacting that responsibility. Thus, while assessment efforts may 




77 



73 



Handbook on Outcomes Assessment 



Edward A. Morante 



start small, the aim over time is to involve people from across the educational community. 
Faculty play an especially important role, but assessment's questions can't be fully addressed 
without participation by student-affairs educators, librarians, administrators, and students. 
Assessment may also involve individuals from beyond the campus (alumni/ae, trustees, 
employers) whose experience can enrich the sense of appropriate aims and standards for 
learning. Thus understood, assessment is not a task for small groups of experts but a 
collaborative activity; its aim is wider, better-informed attention to student learning by all 
parties with a stake in its improvement. 

7. Assessment makes a difference when it begins with issues of use and illuminates 
questions that people really care about. Assessment recognizes the value of information in 
the process of improvement. But to be useful, information must be connected to issues or 
questions that people really care about. This implies assessment approaches that produce 
evidence that relevant parties will find credible, suggestive, and applicable to decisions that 
need to be made. It means thinking in advance about how the information will be used, and 
by whom. The point of assessment is not to gather data and return "results"; it is a process 
that starts with the questions of decision-makers, that involves them in the gathering and 
interpreting of data, and that informs and helps guide continuous improvement. 

8. Assessment is most likely to lead to improvement when it is part of a larger set of 
conditions that promote change. Assessment alone changes little. Its greatest contribution 
comes on campuses where the quality of teaching and learning is visibly valued and worked 
at. On such campuses, the push to improve educational performance is a visible and primary 
goal of leadership; improving the quality of undergraduate education is central to the 
institution's planning, budgeting, and personnel decisions. On such campuses, information 
about learning outcomes is seen as an integral part of decision making, and avidly sought. 

9. Through assessment, educators meet responsibilities to students and to the public. 

There is a compelling public stake in education. As educators, we have a responsibility to the 
publics that support or depend on us to provide information about the ways in which our 
students meet goals and expectations. But that responsibility goes beyond the reporting of 
such information; our deeper obligation — to ourselves, our students, and society — is to 
improve. Those to whom educators are accountable have a corresponding obligation to 
support such attempts at improvement. 
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